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Preface 


The  objective  of  the  research  effort  described  in  this  report  is  to  quantify  the  marginal  impact 
of  networking  as  part  of  an  effort  to  evaluate  the  concept  of  network-centric  operations.  Spe¬ 
cifically,  this  report  analyzes  networking  concepts  and  uses  simulation  results  from  agent- 
based  combat  models  to  quantify  and  assess  the  marginal  benefit  of  networking  concepts 
with  respect  to  warfighter  effectiveness  at  the  tactical  level.  It  will  give  the  Army  unique  and 
relevant  information  to  guide  its  transition  to  a  new  force  makeup  that  will  be  knowledge- 
based  and  network-centric. 

This  work  is  in  support  of  a  project  entitled  “Exploring  the  Capability  of  Network- 
Based  Operations,”  which  is  being  co-sponsored  by  the  Chief  Information  Officer  (G-6)  of 
the  United  States  Army  and  the  Office  of  the  Assistant  Secretary  of  the  Army  for  Acquisi¬ 
tion,  Logistics,  and  Technology.  A  key  task  for  this  project  is  to  “quantify  the  impact  of  cog¬ 
nitive  capability  on  the  performance  of  network-centric  operations.”  This  report  addresses 
this  task  appropriately. 

This  research  was  conducted  in  RAND  Arroyo  Center’s  Force  Development  and 
Technology  Program.  RAND  Arroyo  Center,  part  of  the  RAND  Corporation,  is  a  federally 
funded  research  and  development  center  sponsored  by  the  United  States  Army. 

The  Project  Unique  Identification  Code  (PUIC)  for  the  project  that  produced  this 
document  is  DAPRR05030. 
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For  more  information  on  RAND  Arroyo  Center,  contact  the  Director  of  Operations 
(telephone  310-393-0411,  extension  6419;  FAX  310-451-6952;  email  Marcy_Agmon@ 
rand.org),  or  visit  Arroyo’s  web  site  at  http://www.rand.org/ard/. 
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Summary 


The  concept  of  network-centric  operations  (NCO)  was  described  by  the  GAO  (2004)  as 
follows: 1 

The  emerging  concept  of  networked  operations,  referred  to  by  DoD  as  network¬ 
centric  operations  [NCO],  involves  developing  communications  and  other  linkages 
among  all  elements  of  the  force  to  create  a  shared  awareness  of  operations. 

The  objective  of  the  research  effort  described  in  this  report  is  to  quantify  the  mar¬ 
ginal  impact  of  networking  as  part  of  an  effort  to  evaluate  the  concept  of  NCO.  Three  sets  of 
capabilities  initially  identified  as  relevant  are 

•  Sense/acquire  data. 

•  Disseminate  and  communicate  data. 

•  Interpret,  fuse,  and  react  to  the  data. 

These  capabilities  correspond  to  the  sensing,  communication,  and  cognitive  factors,  respec¬ 
tively,  that  are  analyzed  in  this  work.  In  addition,  we  observed  that  the  force  makeup  is  an 
additional  factor  that  has  a  marginal  impact  along  with  these  three  sets  of  factors. 

In  this  report,  we  take  our  definition  of  “cognitive”  from  Gartska  (2000),  who  de¬ 
fines  a  “cognitive  domain”  as 

the  mind  of  the  warfighter  and  the  supporting  populace.  This  is  the  domain  where 
battles  and  wars  are  won  and  lost.  This  is  the  domain  of  intangibles:  leadership,  mo¬ 
rale,  unit  cohesion,  level  of  training  and  experience,  situational  awareness,  and  pub¬ 
lic  opinion.  This  is  the  domain  where  tactics,  techniques,  and  procedures  reside. 

This  is  broad  and  difficult  to  quantify.  In  this  report,  the  focus  is  narrowed  to  a  small  but 
critical  subset  of  cognitive  parameters,  namely,  accuracy  of  assessments  of  available  targets 
and  the  rate  at  which  they  can  be  prosecuted  (we  call  these  collectively  “targeting  ability”). 
One  specific  objective  of  this  report  is  to  discuss  the  relative  impact  of  all  of  the  aforemen¬ 
tioned  factors  on  overall  warfighter  effectiveness. 

A  multi-agent-based,  force-on-force  simulator  tool  called  Map  Aware  Non-Uniform 
Automata  (MANA)  was  used  to  evaluate  warfighter  effectiveness  for  a  simple  urban  scenario. 


'Alberts,  Garstka,  and  Stein  (1999)  specifically  define  NCO  as  “the  information  superiority-enabled  concept  of  operations 
that  generates  increased  combat  power  by  networking  sensors,  decision  makers,  and  shooters  to  achieve:  shared  awareness, 
increased  speed  of  command,  a  higher  tempo  operations,  and  greater  lethality.” 
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Agent-based  simulations  (ABS)  utilize  little  to  no  scripting  of  movements  and  are  made  up  of 
multiple  agents  and  objects  that  behave  autonomously. 

Tens  of  thousands  of  MAN  A  runs  were  conducted  in  an  attempt  to  examine  the  im¬ 
pact  of  varied  cognitive,  communication,  and  sensing  factors  on  warfighter  effectiveness  us¬ 
ing  a  data  farming  process.  Statistical  analysis  for  numerous  simulation  results,  as  part  of  the 
data  farming  exercise,  quantified  the  correlation  between  the  factors  discussed  above  and 
warfighter  effectiveness. 

In  this  report,  we  explicitly  incorporated  one  of  the  costs  of  networking  by  modeling 
the  capacity  limits  of  a  communication  channel  as  a  result  of  congestion.  One  clear  conclu¬ 
sion:  Warfighter  effectiveness  was  affected  by  many  of  the  parameters  considered,  including 
the  parameter  called  communication  capacity.  Specifically,  effectiveness,  as  measured  by  a 
loss  ratio,  could  be  cut  in  half  without  sufficient  capacity;  message  latency  (delay)  affected 
warfighter  effectiveness  by  as  much  as  50  percent  for  a  given  capacity  in  a  selected  scenario. 
On  the  other  hand,  improvements  in  effectiveness  brought  about  by  increasing  communica¬ 
tion  capability  eventually  diminish.  We  also  observed  that  while  network  capability  may 
sometimes  increase  warfighter  effectiveness,  sometimes  the  force  makeup  is  insufficient  to 
support  improvements  in  effectiveness.  By  some  measures  (loss  ratio,  or  LR),  increased  net¬ 
working  capability  worsened  the  outcome. 

Agent-based  simulation  is  relatively  new  in  terms  of  utilization  for  analyses  of  force- 
on-force  combat  scenarios.  Certainly,  more  validation  will  be  needed.  Furthermore,  the  re¬ 
sults  in  this  report  may  be  sensitive  to  the  assumptions  that  are  made  with  regard  to  network 
architecture,  force  makeup,  and  technology  performance.  Although  a  large  space  of  possibili¬ 
ties  was  explored  for  the  parameters  under  consideration,  all  results  need  to  be  taken  in  the 
context  of  the  specific  scenarios  simulated.  Nonetheless,  the  results  show  how  agent-based 
tools  can  be  exploited  to  quantify  the  marginal  impact  of  networks  and  networking  perform¬ 
ance  to  warfighters. 
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CHAPTER  ONE 


Introduction 


The  objective  of  the  research  effort  described  in  this  report  is  to  quantify  the  marginal  impact 
of  networking  as  part  of  an  effort  to  evaluate  the  concept  of  network-centric  operations 
(NCO).  The  GAO  (2004)  provides  a  description  of  NCO  as  follows:1 

The  emerging  concept  of  networked  operations,  referred  to  by  DoD  as  network¬ 
centric  operations  [NCO],  involves  developing  communications  and  other  linkages 
among  all  elements  of  the  force  to  create  a  shared  awareness  of  operations. 

We  consider  three  sets  of  parameters  of  networking  (or  network-centric  operations) 
as  a  means  to  investigate  the  impact  of  networking  on  warfighter  effectiveness  (see  Figure 
1.1).  We  identify  these  components  as  follows: 

•  Disseminate  and  communicate  data  (communication  capability). 

•  Sense/acquire  data  (sensing  capability). 

•  Interpret,  fuse,  and  react  to  the  data  (cognitive  capability). 

These  three  capabilities  are  critical  to  the  elements  in  the  “info  grid”  that  Cebrowski  and 
Garstka  (1998)  use  to  illustrate  their  vision  of  NCO.  Their  vision:  All  users  (sensors,  shoot¬ 
ers,  commanders,  etc.)  are  interconnected  as  part  of  an  information  grid  (see  Figure  1.2). 

Figure  1.1 

Capabilities  That  Affect  Networking  Performance 


Cognitive  f 


Sensing 


‘‘Communication  ” 


'Alberts,  Garstka,  and  Stein  (1999)  specifically  define  NCO  as  “the  information  superiority-enabled  concept  of  operations 
that  generates  increased  combat  power  by  networking  sensors,  decision  makers,  and  shooters  to  achieve:  shared  awareness, 
increased  speed  of  command,  a  higher  tempo  operations,  and  greater  lethality.” 


1 


2  The  Impact  of  Network  Performance  on  Warfighter  Effectiveness 


Figure  1.2 

For  NCO:  An  Information  Grid  Connects  the  Distributed  Sensors 
and  Shooters  Across  the  Battlespace 


INFO  GRID 


RAND  M0156-1.1 


SOURCE:  Cebrowski  and  Garstka  (1998). 


One  objective  of  this  report  is  to  discuss  the  relative  impact  of  these  components  on 
overall  warfighter  effectiveness.  The  cognitive  capability,  as  a  component  of  networking  per¬ 
formance  (or  the  effectiveness  of  NCO),  is  arguably  the  most  difficult  to  quantify  and  test.  It 
is  also  critical  for  effective  command  and  control  (C2),  especially  with  regard  to  determining 
what  is  or  is  not  a  target  and  the  speed  at  which  that  can  be  done.  The  cognitive  component 
may  also  be  the  one  with  the  largest  impact.  Quoting  Baker  (2002):  “The  speed  of  informa¬ 
tion  transfer  in  Network  Centric  Warfare  is  rapidly  outpacing  the  human  capacity  to  absorb 
and  act  effectively.”  A  GAO  report  (2004)  supports  this  observation  based  on  Operation 
Iraqi  Freedom  (OIF)  after-action  reports  (which  we  will  discuss  later).  And  as  we  will  report, 
the  force  makeup  affected  the  marginal  impact  of  the  three  aforementioned  sets  of  factors. 


The  Advantages  of  Agent-Based  Modeling 

Simulations  with  agent-based  models  (ABM)  are  made  up  of  agents  and  objects  that  behave 
autonomously.  Liu  (2001)  defines  agents  to  be  a  “wide  spectrum  of  computational  entities 
that  can  sense  their  local  task  conditions  and  make  decisions  on  how  to  react  to  sensed  con¬ 
ditions  by  performing  certain  behaviors  in  task  environments.”  As  defined  by  Phillips 
(2004),  ABM  are  simulations  that  represent  autonomous  agents  that  make  decisions  based 
on  data  received  through  organic  sensors,  or  communications  links  (the  “network”).  Simula¬ 
tions  with  ABM  typically  utilize  little  to  no  scripting  of  movements.2  Users  of  agent-based 


2  In  some  simulation  tools  like  MANA,  optional  waypoints  can  be  specified  that  compose  a  path  with  a  location  goal. 
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simulation,  which  include  the  United  States  Marine  Corps  Warfighting  Lab,3  value  it  be¬ 
cause,  among  other  possibilities,  it  is  seen  as  a  means  to  model  the  human  intangibles  of 
combat,  e.g.,  trust,  unit  cohesion,  fatigue,  morale,  and  leadership.  Complex  behavior  such  as 
swarming  (Arquilla  and  Ronfeldt,  2000)  can  emerge  from  this  approach. 

Cianciolo  (2003)  rightly  asserts,  “In  order  to  fully  exploit  and  implement  the  concept 
of  Network  Centric  Warfare,  advocates  must  take  into  consideration  the  behavior  of  men  in 
battle  and  recognize  that  external  and  situational  factors  such  as  fear,  danger,  stress,  and 
hardship  will  impact  on  the  decisions/judgment  of  leaders  and  individual  warfighters  alike.” 
Russell,  Russell,  and  Benke  (1996)  suggest  that  this  is  rarely  done  at  present  when  they  note 
that  “quantitative  evaluations  of  combat  performance  rarely  include  detailed  information  on 
human  factors.”  With  respect  to  these  factors,  agent-based  models  that  incorporate  personal¬ 
ity  traits  can  be  useful. 

There  exist  ABM  that  address  these  factors  by  enabling  the  user  to  model  considera¬ 
tions  such  as  the  “cognitive  domain”  and  other  human  factors  quickly,  efficiently,  and 
autonomously.  Agents  in  the  model  interact  with  each  other  in  nonlinear  ways,  and  they 
adapt  to  their  local  environment.  How  they  cooperate  in  the  battlespace  is  extracted  from 
their  personality,  which  is  shaped  by  weighting  factors  that  control  their  propensity  to  move 
toward  or  away  from  different  units  or  objectives.  The  environment  in  which  the  agent  be¬ 
longs  can  be  set  up  to  represent  network-centric  issues  in  such  a  way  that  their  actions  de¬ 
pend  on  the  quality,  or  lack  thereof,  of  the  information  they  receive.  Knowing  this,  complex 
topics,  such  as  unit  cohesion  and  dissemination  of  information,  can  not  only  be  explored  as 
concepts  but  also  quantified  in  terms  of  their  impact. 

Map  Aware  Non-Uniform  Automata  (MANA) 

MANA4  was  developed  by  the  New  Zealand  Defence  Technology  Agency.  It  is  an  agent- 
based  simulation.  In  MANA,  knowledge  of  enemy  contacts  (and  friendly  locations)  is  re¬ 
ferred  to  as  situational  awareness.  Two  types  of  situational  awareness  (SA)  maps  are  provided 
in  MANA:  a  squad  map,  which  holds  direct  squad  contact  memory,  and  an  inorganic  map, 
which  stores  contact  memories  provided  by  other  squads  through  communications  links. 
MANA  allows  communication  of  contact  sightings  between  squads.  An  extensive  range  of 
parameters  allows  issues  involving  communications  links  to  be  thoroughly  explored.  MANA 
contains  terrain  features,  such  as  roads,  that  agents  can  follow  and  undergrowth  that  agents 
can  use  for  concealment.  Users  can  define  a  set  of  waypoints  for  agent  movements  and  not 
just  specify  an  ultimate  objective  for  autonomous  movement.  Events,  such  as  being  shot  at, 
taking  a  shot,  reaching  a  waypoint,  and  making  enemy  contact,  can  all  trigger  a  different  per¬ 
sonality  set,  which  lasts  for  a  set  time.  Personality  changes  can  apply  to  individuals  or  to  a 
whole  squad  at  once.  According  to  Lucas,  Sanchez,  and  Cioppa  (2004),  agent-based  simula¬ 
tions  have  the  potential  to  look  at  intangibles  like  leadership,  courage,  fear,  and  aggressive¬ 
ness.  Situational  awareness  in  MANA  is  characterized  by  SA  maps  that  contain  only  targeting 
and  “Blue  force”  information  and  battle  damage  assessments. 


3  http://www.mcwl. quantico. usmc.mil/divisions/DivisionDescription. cfm?ID=23. 

4  Pronounced  Mah-Nuh  with  emphasis  on  neither  syllable,  “MANA”  is  a  Maori  word  meaning  an  aura  of  respect  and 
authority. 
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Factoring  Cognitive  Parameters  with  MANA 

In  MANA,  individual  agents  have  their  own  local  perspective  on  their  environment.  This  is 
achieved  as  follows:  Agents  in  MANA  have  individual  personality  weights  that  can  be  ex¬ 
ploited  to  factor  the  aforementioned  parameters  (Galligan,  Anderson,  and  Lauren,  2004). 
Personality  settings  affect  mobility,  targeting,  clustering,  and  other  agent  decisions.  These 
personality  settings  can  be  made  dynamic,  i.e.,  they  can  be  programmed  to  change  based  on 
external  events  or  activity  types  or  agent  status  (casualty  status,  fuel  status,  ammunition 
status)  or  mobility  (e.g.,  moving  or  stationary). 

With  regard  to  mobility,  an  agent  can  be  assigned  varying  amounts  of  attraction  or 
repulsion  uniquely  associated  with  either  friends,  foes,  neutrals,  unknown  agents,  terrain 
types  (concealed  areas,  open  areas),  and  specified  objective  areas.  Varying  amounts  of  clus¬ 
tering  or  swarming  may  be  a  result.  Agents  are  also  assigned  targeting  decisionmaking 
schemes,  which  can  be  programmed  to  be  different  for  various  classes  of  targets.  Autono¬ 
mous  mobility  is  assigned  based  on  either  specific  movement  algorithms,  e.g.,  Gill  (2004), 
Gill  and  Grieger  (2003),  and  Lauren  and  Stephen  (2000),  or  some  predetermined  path,  or  a 
combination.  Further  discussion  is  provided  in  the  following  section. 

Agent  Movement  in  MANA 

The  movement  of  an  agent  in  MANA,  as  the  simulation  progresses  through  steps,  involves 
penalty  calculations.  This  approach  shares  some  similarities  with  robotic  movement  algo¬ 
rithms  that  determine  the  path  of  least  resistance,  but  there  is  more  randomness  incorpo¬ 
rated. 

Specifically,  there  are  multiple  equations  that  calculate  a  penalty  for  each  agent’s  pos¬ 
sible  movement  to  a  neighboring  cell.  Based  on  their  values  and  the  agent’s  personality 
weight  vector,  movement  is  determined.  Like  other  cellular  automaton  (CA)  models  (Illa- 
chinski,  2001),  the  movement  neighborhood  is  defined  as  the  cells  surrounding  the  agent 
(see  Figure  1.3).  The  actual  move  is  the  one  that  incurs  the  least  penalty.  MANA  allows  op¬ 
tional  specification  of  waypoints  and  objective  locations  that  adds  a  degree  of  “far¬ 
sightedness.” 

Figure  1.3 

CA  Model  Neighborhood 


Factors  Under  Consideration 

A  number  of  factors  were  varied  in  the  investigations  described  in  this  report.  We  assign 
them  to  the  three  areas  of  impact,  (i)  communication  capability,  (ii)  sensing  capability,  and 
(iii)  a  subset  of  cognitive  parameters  we  refer  to  as  targeting  capability.  We  elaborate  on  them 
as  follows. 
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Communication  Representation 

MANA  incorporates  a  number  of  specific  communication  capabilities  associated  with  links. 
They  include: 

1 .  reliability 

2.  capacity,  and 

3.  latency. 

Reliability  is  the  likelihood  that  a  given  message  will  be  successfully  transmitted  on  a 
communications  link  per  try.  Thus,  it  is  similar  to  a  message  completion  rate.  Attempts  will 
be  made  at  resending  unsuccessful  messages  until  they  are  successfully  communicated.  Reli¬ 
ability  ranges  from  0  percent  to  100  percent. 

Capacity  is  the  number  of  messages  that  can  be  sent  through  the  link  per  time  step. 
Latency  is  the  number  of  time  steps  taken  for  each  message  to  reach  the  receiving  squad. 

Sensing  Representation 

In  MANA,  sensing  ability  is  accounted  for  by  the  detection  range.  This  is  the  radius  in  cells 
that  an  agent  can  see  targets  in  its  environment.  Detection  does  not  imply  classification  or 
identification.  Contacts  are  recorded  as  unknown  on  the  situational  awareness  maps  if  they 
are  recorded  in  an  area  where  the  detection  range  extends  but  the  classification  range  does 
not.5 

Cognitive  Representation 

Kamradt  (2003)  provides  a  concise  summary  of  the  limitations  within  the  cognitive  realm: 

Compared  to  the  1  Os  of  gigabits  per  second  that  we  might  someday  push  across  our 
networks,  experts  believe  the  limits  of  rhe  human  mind  are  far  more  restrictive.  Ex¬ 
perimentation  indicates  that  while  our  senses  take  in  huge  amounts  of  information, 
consciousness  processes  only  about  40  bits  per  second — at  most.  Additional  research 
indicates  that  people  are  far  less  efficient  at  multitasking  than  was  once  believed.  It 
seems  trying  to  do  several  things  at  once  or  in  rapid  succession  takes  longer  than 
doing  them  one  at  a  time  and  may  leave  less  brainpower  to  perform  each  task  .  .  . 

The  bottom  line  is  that  people  are  easy  to  overload. 

In  this  report,  the  focus  is  narrowed  to  a  small  but  critical  subset  of  cognitive  parame¬ 
ters,  namely,  accuracy  of  assessments  of  available  targets  and  the  rate  at  which  they  can  be 
prosecuted  (we  call  these  collectively  “targeting  ability”  or  “targeting  capability”).  Targeting 
capability  captures  the  limits  discussed  above  by  Kamradt.  Two  corresponding  MANA  pa¬ 
rameters  are  used  to  account  for  targeting  capability  (which  is  an  ability  that  partly  results 
from  good  cognition). 

Why  have  an  accuracy  parameter?  Cianciolo  (2003)  cautions  “NCW  [network¬ 
centric  warfare]  advocates  must  address  the  issue  of  accuracy,  especially  with  regard  to  highly 
perishable  information.  History  has  demonstrated  that  combat  information  can  be  extremely 


5  Checking  the  “Lock  to  Classification  Range”  option  in  the  graphical  user  interface  (GUI)  ensures  that  detections  are  al¬ 
ways  classifiable  by  synchronizing  the  detection  and  maximum  classification  ranges.  In  MANA,  its  values  range  between  0 
and  1000. 
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inaccurate,  especially  when  individuals  are  influenced  by  fatigue,  isolation,  and  stress.”  In  our 
work,  that  caution  is  heeded  by  including  it  as  a  factor. 

In  MANA,  the  accuracy  parameter  sets  the  probability  that  a  contact’s  type  will  be 
passed  correctly.  When  a  link  is  acting  inaccurately,  an  incorrect  type — out  of  the  pool  of 
enemy,  friend,  neutral,  and  unknown — is  sent  for  the  contact.  The  accuracy  parameter  is 
particularly  useful  for  friendly  fire  type  studies.  Its  values  are  between  0  percent  and  100  per¬ 
cent.  An  accuracy  of  0  percent  means  the  link  always  sends  the  incorrect  contact  type  and 
100  percent  means  the  link  always  sends  the  correct  contact  type. 

The  number  of  targets  within  both  sensor  and  firing  range  that  can  be  engaged  in  a 
single  time  step  is  referred  to  here  as  target  rate.  The  actual  number  entered  in  the  simulation 
is  between  0  and  1000,  where  a  value  of  150  means  100  percent  chance  of  engaging  one  tar¬ 
get,  and  a  further  50  percent  chance  of  engaging  another  target  in  a  step. 


Relevant  Studies  Using  MANA 

Ipekci  and  Lucas  (2002)  at  the  Naval  Postgraduate  School  documented  their  research  effort 
using  MANA  in  the  monograph  titled  “How  Agent-Based  Models  Can  Be  Utilized  to  Ex¬ 
plore  and  Exploit  Non-Linearity  and  Intangibles  in  Guerilla  Warfare.”  This  study  was  con¬ 
cerned  with  a  small  unit  operation  where  intangibles  like  unit  cohesion,  movement,  and  de¬ 
cisionmaking  at  the  level  of  individual  combatants  were  of  interest.  MANA  is  capable  of 
assessing  these  types  of  factors.  The  study’s  conclusions  are  as  follows:  (1)  Results  are  mostly 
affected  by  factors  associated  with  Red  (the  opposing  force);  stealth  ability  is  important.  (2) 
More  cohesive  guerilla  forces  that  do  not  stay  with  the  injured  and  form  big  groups  do  better 
at  infiltration.  (3)  Red  negated  Blue  firepower  by  increasing  the  sizes  of  infiltration  teams. 

Dekker  (2002)  uses  force-on-force  simulation  experiments  to  examine  the  marginal 
impact  of  C4ISR.6  He  develops  an  expression  that  relates  the  connectivity  of  a  network  to¬ 
pology  and  a  measure  of  intelligence  to  the  adjusted  loss  exchange  ratio  (ALER).  The  expres¬ 
sion,  developed  from  a  set  of  simulation  experiments,  follows: 

ALER^1.6^7(1.043)k, 

where  K  is  the  node  connectivity.  The  coefficient  /  is  called  the  intelligence  quotient. 

The  intelligence  coefficient  /  is  obtained  by  summing  (over  all  combat  nodes  and  all 
relevant  sensors  for  that  node)  the  quotient  of  sensor  quality  and  total  path  delay.  If  Ay  is  the 
total  path  delay  from  sensor  node  i  to  combat  nodey  (or  infinite  if  there  is  no  connection), 
and  qj  is  the  quality  of  sensor  I,  then 


i  =  Y 
L-‘i  A; 


6  Dekker’s  work  simulated  10  units  engaged  in  combat  with  a  non-networked  enemy  force  of  30  units.  Friendly  force 
makeup  included  two  long-range  sensors,  two  headquarters  units,  and  six  combat  units  (which  also  had  limited  sensing 
capabilities).  MANA  was  used  as  the  agent-based  combat  simulator. 
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This  reference  does  not  address  the  method  used  for  assigning  a  quality  to  a  sensor. 
Essentially  the  intelligence  coefficient  measures  the  ability  of  the  network  to  effectively  move 
sensor  information  to  the  point  where  it  is  needed. 

Lauren,  Stephen,  and  Hore  (2003)  used  MANA  to  model  operational  questions  for 
East  Timor.  The  simulations  were  in  part  used  as  an  effort  to  see  how  suitable  MANA  could 
be  as  a  tactical  decision  support  tool.  In  that  study,  the  authors  were  charged  with  addressing 
two  key  questions:  What  was  the  most  effective  way  to  conduct  a  cordon-and-search  opera¬ 
tion  on  a  designated  village?  Was  it  possible  to  show  the  benefit  of  the  use  of  “trackers”  in 
this  theater  of  operations? 

The  modeling  results  were  considered  a  priori  and  compared  with  decisions  and  ob¬ 
servations  actually  made  by  tactical  commanders  for  actual  operations.  Post-operation  review 
seemed  to  validate  the  MANA  results  using  data  and  observations  by  tactical  commanders 
deployed  to  East  Timor. 

The  operation  that  was  analyzed  is  described  as  an  effort  to  capture  an  individual 
hiding  in  a  village.  Information  flow  between  the  target  and  sympathetic  bystanders  was 
modeled.  An  infantry  company  approaches  the  village  in  three  groups  from  three  directions 
in  the  daylight.  An  operation  is  considered  successful  if  the  target  is  captured.  One  interest¬ 
ing  observation  was  made  with  respect  to  the  impact  of  an  additional  aerial  observer  sensor. 
Quoting  Lauren,  Stephen,  and  Hore  (2003): 

Although  the  addition  of  the  aerial  observer  significantly  increased  the  number  of 
Red  that  Blue  was  able  to  catch,  it  also  increased  the  number  of  Blue  casualties.  This 
was  because  the  effect  of  the  addition  of  an  observer  was  to  increase  the  chances  of 
an  encounter  between  the  sides,  hence  increasing  the  likelihood  of  casualties. 

These  authors  concluded  that  “technological  fixes  do  not  necessarily  make  greater  gains  in  an 
operation  than  the  choice  of  appropriate  tactics.” 


Organization  of  This  Report 

The  remainder  of  this  report  is  organized  as  follows:  Chapter  Two  provides  a  summary  of 
the  concept  of  network-centric  operations,  along  with  critiques  of  the  concept. 

Chapter  Three  discusses  the  results  of  force-on-force  simulation  experiments  using 
MANA,  which  were  run  to  investigate  the  marginal  impact  of  various  factors  that  affect  war¬ 
fighter  effectiveness.  Results  for  four  sets  of  simulation  experiments  are  described. 

The  first  set  of  experiments  has  two  Blue  force  squads,  one  an  indirect-fire  unit  and 
the  other  an  infantry  squad.  The  only  messages  that  get  passed  are  “calls  for  fire”  from  the 
infantry  squad  back  to  the  indirect-fire  squad.  These  experiments  show  the  diminishing  re¬ 
turn  from  increased  communication  capacity. 

A  second  set  is  run  using  the  same  scenario,  but  a  dynamic  communication  model  is 
coded  into  the  force-on-force  simulator;  this  allows  MANA  to  vary  communication  reliabil¬ 
ity  based  on  message  traffic  and  channel  capacity  during  each  time  step.  The  main  result  is 
the  same  as  the  first  simulation  set,  except  effectiveness  is  lower  overall.  So  these  runs,  com¬ 
pared  with  the  first  set  of  experiments,  show  how  static  assumptions  on  communications 
ability  affect  outcome. 
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The  third  set  of  simulation  runs  is  executed  with  an  expanded  scenario;  a  second 
squad  of  six  infantry  is  added  to  the  initial  scenario  to  give  us  a  better  understanding  of  the 
marginal  impact  of  communications  and  sensing. 

The  fourth  set  of  simulation  runs  came  out  of  work  done  at  the  9th  Project  Albert 
International  Workshop  in  Wellington,  New  Zealand.  A  new  scenario  is  used  in  which  Blue 
forces  are  attempting  to  secure  an  area  around  a  mosque  and  stop  Red  forces  from  entering. 
The  result  observed  was  that  the  Blue  force  consistently  fared  poorly,  despite  increased  net¬ 
working  capability,  until  the  force  was  augmented  with  sufficient  indirect-fire  capability. 
This  scenario  and  the  resulting  experiments  were  run  to  provide  counterexamples  to  unquali¬ 
fied  assertions  about  the  benefits  of  networking  (that  do  not  consider  other  factors,  e.g.,  force 
makeup,  scenario,  etc.). 

Finally,  Chapter  Four  summarizes  the  overall  conclusions  of  the  report.  Appendix  A 
provides  details  on  the  models  of  warfighter  effectiveness  that  are  synthesized  and  examined 
in  Chapter  Three.  Appendix  B  summarizes  past  work  on  communication  models  leveraged 
in  this  report. 


CHAPTER  TWO 


Evaluating  the  Concept  of  Network-Centric  Operations 


The  multilayered  concept  of  future  force  networking  (as  shown  in  Figure  2.1)  is  complex. 
The  key  motivation  of  this  report  is  to  be  able  to  quantitatively  assess  the  marginal  impact  of 
networking,  which  we  attempt  by  considering  each  of  the  components  of  networking  per¬ 
formance  outlined  in  Chapter  One. 

Figure  2.1 

Future  Networks  Will  Be  Multilayered 


Quantifying  the  Benefits  of  NCO 

Proponents  cite  the  potential  for  wide  gains  in  warfighter  effectiveness.  In  fact,  interpreta¬ 
tions  of  Army  doctrine  by  some  (Mazzanti,  2001,  Gumbert  et  al,  2003,  TRADOC,  1995) 
are  that  the  impact  of  information  on  the  warfighter,  provided  by  ubiquitous  networks,  is 
exponential,  and  they  explicitly  formulate  this  notion1  as  follows: 


1  Quoting  (Gumbert  et  al,  2003):  “COMBAT  POWER  is  defined  as  a  linear  function,  being  the  sum  of  maneuver,  fire¬ 
power,  and  protection  multiplied  by  leadership.  In  the  future  combat  systems  (FCS)-equipped  force,  combat  power  be- 
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Combat  power  = 

((Maneuver  +  Firepower  +  Protection)*Leadership)Informat,on 

One  of  the  supposed  underpinnings  of  this  conjecture  is  Metcalfe’s  law.* 2  Gilder  (1993)  in¬ 
terprets  Metcalfe’s  law  as  follows: 

Metcalfe’s  law  of  telecosms:3  connect  any  number  “n”  of  machines — whether  com¬ 
puters,  phones  or  even  cars — and  you  get  “n”-squared  potential  value. 

This  is  derived  from  the  calculation  of  the  maximum  number  of  possible  (pair-wise) 
connections,  e.g.,  n(n— 1)/2,  in  an  information-providing  network  of  n  nodes.  Reed’s  law 
(2000),  which  claims  greater  gains  (e.g.,  exponential),  is  also  derived  in  part  by  a  similar  type 
of  calculation  but  includes  all  the  n- way  (e.g.,  3-way,  4-way,  etc.)  connections.  An  example 
of  an  extrapolation  of  this  law  to  the  military  environment  is  shown  in  Figure  2.2,  most  re¬ 
cently  cited  by  Fisher  (2003). 

There  has  been  renewed  debate  over  the  meaning  and  appropriateness  of  Metcalfe’s 
law  in  a  military  context.  Giffin  and  Reid  (2003)  argue  that  it  has  been  inappropriately  in¬ 
terpreted.  Specifically,  they  argue  that  NCO  proponents  wrongly  interpret  Metcalfe  to  mean 
an  increase  in  the  power  of  the  network  or  the  value  of  its  transactions.  They  claim  instead 


Figure  2.2 

Conjecture  on  Impact  of  Networks  (Connections)  on  Warfighter  Effectiveness 


comes  an  exponential  equation  where  the  power  of  information  will  raise  the  factors  of  maneuver,  firepower,  protection, 
and  leadership.  Therefore,  information  and  intelligent  command  and  control  (C2)  systems  are  key  to  the  success  of  the 
FCS-equipped  force.  This  C2  system  must  enable  the  FCS-equipped  force  to  synchronize  intelligence,  maneuver,  effects, 
and  logistics,  as  well  as  the  exchange  of  information  with  joint  or  combined  task  force  C2  systems  and  the  Army's  tactical 
C2  systems.  In  short,  the  C2  system  will  enable  a  force  that  is  both  network-  and  execution-centric  to  employ  combined 
arms  and  joint  capabilities  at  the  lowest  tactical  echelons.” 

2  Robert  Metcalfe  is  the  inventor  of  Ethernet  LANs  and  founder  of  3Com. 

3  George  Gilder  and  others  use  the  term  “telecosm”  when  referring  to  “the  world  enabled  and  defined  by  new  communica¬ 
tions  technology.” 
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that  it  only  applies,  at  best,  to  goods  and  services  necessary  to  participate  in  a  network.  For 
example,  facsimile  machines  may  be  increasingly  more  valuable  as  more  people  acquire  them. 


NCO  and  the  Cost  of  Communication 

Whatever  the  potential  benefit  accrued  from  a  network,  the  costs  of  such  networking  have  to 
be  factored  to  provide  a  perspective  on  true  value.  Tactical  Army  units  will  require  mobile, 
ad  hoc  (wireless)  communication.  In  wireless  networks,  communication  costs  can  actually 
increase  with  the  size  (n)  of  the  network.  If  you  assume  the  network  value  is  proportional  to 
network  throughput  (measured  in  bits  per  second),  we  see  then  from  Gupta  and  Kumar 
(2000)  and  others  that  at  least  one  measure  of  network  value  (e.g.,  throughput)  decreases 
nonlinearly,  as  shown  in  Figure  2.3.  Certainly,  without  sufficient  throughput,  information 
cannot  be  passed. 

As  shown  in  Figure  2.3,  Gupta,  Gray,  and  Kumar  (2001)  report  wireless  networking 
experiments  in  which  per-node  throughput  decays  like  c/n L68.  If  network  value  increases  pro¬ 
portional  to  n{n—\)l2  but  decreases  proportional  to  ?A68  or  sqrt(nlogfiz)),  then  the  aforemen¬ 
tioned  quantifications  of  the  impact  of  network  connectivity  and  information  on  combat 
power,  as  postulated  by  Fisher  (see  Figure  2.2),  Gumbert  et  al  (2003),  and  the  U.S.  Army 
Training  &  Doctrine  Command  (TRADOC,  1995),  are  in  doubt.  In  other  words,  there  may 
be  nearly  as  much  cost  (or  more)  to  sustaining  a  network  as  there  is  benefit  in  maintaining  it. 
In  the  research  effort  documented  in  this  report,  we  provide  simulations  that  factor  in  the 
scaling  limitation  of  the  wireless  networks  that  are  envisioned  at  the  tactical  level. 


Figure  2.3 

Impact  of  Scaling  on  Useable  (Per-node)  Capacity 
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Proponents  of  NCO  argue  that  its  key  attribute  is  the  potential  to  connect  to  any¬ 
one,  not  necessarily  constant  or  simultaneous  use  of  all  links.  In  other  words,  the  network 
doesn’t  have  to  be  fully  connected  at  all  times.  Rather,  it  needs  to  be  able  to  connect  anyone 
at  any  time  (McEver,  2004).  This  poses  a  difficulty  for  analysts.  Exploratory  analysis  needs  to 
examine  how  a  given  network  could  be  potentially  exploited  by  the  warfighter.  This  will  be 
very  difficult  for  a  scripted  simulation.  It  is  also  difficult  for  real-world  experimentation, 
which  can  only  be  repeated  a  small  number  of  times.  Some  argue  that  only  numerous  sto¬ 
chastic  simulations  are  appropriate  to  study  these  kinds  of  effects  (Lucas,  2000),  although  the 
exact  numbers  of  simulations  depend  on  the  underlying  variance  of  the  statistics  used  and 
the  degree  of  accuracy  required.4  The  research  effort  reported  here  uses  stochastic,  agent- 
based  simulation  to  account  for  the  dynamics  of  network  use. 


NCO  and  the  Impact  of  Human  Factors  and  Behaviors 

Baker  (2002)  highlights  what  he  sees  as  “major  problems  that  face  NCW  with  regard  to  hu¬ 
man  tendencies.”  They  are  listed  below  to  highlight  the  potential  interdependencies  of  net¬ 
working  concepts,  human  factors,  and  human  behavior. 

1 .  Micromanagement:  the  tendency  for  commanders  to  direct  efforts  at  levels  below 
their  immediate  subordinate  commands. 

2.  Information  overload:  the  result  when  information  is  acquired  at  a  rate  that  im¬ 
pedes  the  information  receiver’s  ability  to  process  it. 

3.  Poor  degradation:  the  inability  to  function  autonomously  if  the  network  is  com¬ 
promised. 

4.  Endurance:  decreased  efficiency  of  the  human  mind  after  long  periods  of  time. 

Cianciolo  (2003)  considers  the  impact  of  human  behavior  on  what  he  sees  as  one  of 
the  intended  consequences  of  network-centric  operations.  Specifically,  he  considers  the  op¬ 
portunity  that  NCO  affords  to  redefine  the  size  of  the  basic  fighting  unit.  His  critique  is  as 
follows: 


Fewer  warfighters  will  result  in  a  condition  of  isolation  for  the  individual  soldier, 
and  as  we  have  seen  through  the  study  of  human  behavior  in  combat,  the  more  iso¬ 
lated  the  individual  warfighter  becomes,  the  less  combat  effective  he  is  because  of  the 
stress  associated  with  the  loss  of  association  and  safety  of  the  group. 

Clearly,  the  concept  of  NCO  at  the  tactical  level  provides  many  areas  of  consideration  in¬ 
volving  what  Gartska  (2000)  calls  the  “the  cognitive  domain.”  Our  simulations  in  the  next 
chapter  include  some  of  these  factors,  e.g.,  ones  that  relate  to  cognitive  capability  and  human 
behavior,  albeit  in  a  very  basic  manner.  The  cognitive  aspects  are  focused  on  targeting  capa¬ 
bility. 


4  Quoting  Lucas  (2000):  “The  nature  of  combat,  along  with  fundamental  mathematical  principles,  implies  that  most  com¬ 
bat  simulations  should  be  stochastic.” 


CHAPTER  THREE 


Simulation  Experiments 


In  the  following  sections  we  will  detail  results  from  a  number  of  experiments  as  part  of  a  data 
farming  effort.1  “Data  Farming  is  a  method  to  address  [a]  decision-maker’s  questions  that 
applies  high  performance  computing  to  modeling  in  order  to  examine  and  understand  the 
landscape  of  potential  simulated  outcomes,  enhance  intuition,  find  surprises  and  outliers, 
and  identify  potential  options”  (source:  www.projectalbert.org).  As  part  of  such  an  effort,  a 
large  number  of  MANA  simulation  runs  can  be  used  to  provide  enough  information  so  that 
an  algebraic  expression  of  the  performance  response  is  formulated  as  a  function  of  the  factors 
of  interest  via  regression  analysis.  This  is  also  called  meta-modeling  (Law  and  Kelton,  1991). 


First  Set  of  Experiments 

To  investigate  the  marginal  impact  of  network  performance  on  warfighter  effectiveness,  four 
sets  of  experiments  were  run. 

Description  of  Scenario 

In  the  first  experiment,  a  simple  scenario  was  developed  to  examine  the  marginal  impact  of 
communication  capacity.  The  force-on-force  simulation  pitted  a  small  Blue  force  consisting 
of  two  squads  against  an  opposing  Red  force  of  100  dismounted  fighters.  In  the  simulation, 
the  fighting  occurs  in  a  stylized  urban  environment. 

In  this  scenario,  a  squad  of  six  Blue  infantry  armed  with  direct-fire  weapons  is  mov¬ 
ing  from  a  position  in  the  northwest  (see  Figure  3.1)  to  the  southeast  corner.  (The  term 
squad  is  used  conceptually  in  MANA  to  represent  any  amount  of  soldiers  sharing  the  same 
characteristics.)  The  members  of  this  squad  are  to  minimize  enemy  contact  but  engage  if 
necessary,  and  based  on  that  contact  find  the  best  way  to  reach  the  waypoint  in  the  south¬ 
east.  They  have  the  support  of  another  squad  with  an  indirect-fire  weapon  with  enough 
range  to  cover  the  entire  city.  The  Red  forces  were  randomly  placed  on  the  map,  and  they 
generally  seek  confrontation  with  the  Blue  forces.  The  purpose  of  this  is  to  show  how  sensing 


1  Quoting  Brandstein,  Friman,  and  Horne  (2001):  “Data  farming  was  first  developed  and  used  at  the  Marine  Corps  Com¬ 
bat  Development  Command  in  late  1997.  It  puts  technology  advances  to  work  to  engage  the  scientific  method.  The  tech¬ 
nique  provides  an  opportunity  to  explore  questions  and  grow  more  data  in  the  areas  of  interest.  This  growth  within  a  par¬ 
ticular  definition  of  a  particular  distillation  might  be  in  the  form  of  more  runs  or  a  different  preparation  of  the  sample  space 
to  include  different  parameters,  finer  gradations  of  parameter  values  or  greater  ranges.  After  the  execution  of  samples  and 
analysis  using  data  visualization  and  search  methods,  the  data  farmer  is  free  to  grow  more  data  in  interesting  areas,  integrate 
with  information  from  other  tools,  prepare  a  different  scenario  using  the  same  distillation,  select  another  distillation  or  em¬ 
ploy  any  combination  of  these  possibilities  that  might  lead  to  progress.” 
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and  communicating  can  impact  the  battle  on  a  fundamental  level,  ignoring  complex  behav¬ 
iors  or  route  planning.  Although  clearly  outmatched  (100  to  7),  if  the  Blue  force  is  conserva¬ 
tive  in  its  approach,  it  is  able  to  reach  the  objective.  Blue  force  benefits  from  an  indirect  pre¬ 
cision  weapon  and  a  communication  link  that  allows  the  Blue  infantry  squad  to  pass  target 
information  back  to  the  indirect- fire  squad.  The  Blue  force  makeup  is  illustrated  in  Figure 
3.2.  The  Red  force  did  not  have  such  capacity. 

Figure  3.1 

Scenario:  Small  Force  Maneuvering  and  Supported  by  Precision-Fire  Area  Weapon 


• Blue  Forces  (7) 


•Two  squads 


• Indirect 
Sqd  (1)  / 


• In  fry  Sqd 
(6) 


Red  Forces 
(100) 


Figure  3.2 

Blue  Force  Communication/SA  Link 
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The  only  form  of  situational  awareness  (SA)  the  indirect-fire  squad  has,  hence  the 
only  reason  it  fires,  comes  from  the  inorganic2  messages  it  receives  from  the  infantry  squad. 
If  messages  are  not  being  received,  then  the  fire  team  will  not  discharge  its  weapon.  A  key 
motivation  for  using  MANA  is  that  agents  in  MANA  react  to  messages  about  the  state  of 
other  agents  and  adjust  their  movement  and  firing  autonomously. 

For  our  purposes,  movement  speed  was  left  at  one  cell  per  simulation  time  step.  We 
calibrate  this  to  real-world  time  by  considering  that  average  human  motion  from  walking  to 
running  is  approximately  1-6  meters  per  second.  We  assume  that  each  cell  is  approximately 
5  meters  in  length,  the  agents  move  one  cell  per  time  step,  and  a  time  step  is  one  second  in 
duration.  Since  our  scenario  environment  was  200  x  200  cells,  the  approximate  size  of  the 
town  is  1  km2.  The  terrain  utilized  is  a  fictitious  urban  environment  and  depicts  a  small  Blue 
force  that  is  maneuvering  with  the  support  of  a  precision-fire  area  weapon. 

For  the  described  scenario,  certain  factors  discussed  in  Chapter  One  were  investi¬ 
gated  using  the  data  farming  approach. 

Overview  of  Data  Farming  Effort 

Factors  in  the  scenario  were  varied  to  assess  the  effect  on  the  scenario  outcome  in  terms  of 
the  loss  ratio  (LR)3  as  part  of  a  data  farming  effort  (Horne,  1997).  These  factors  were:  (a) 
capacity,  which  reflects  the  number  of  messages  that  can  be  transmitted  at  one  period;  (b) 
latency,  which  represents  any  delay  in  terms  of  the  message  that  gets  passed  when  something 
is  sensed  and  this  information  is  communicated  out;  (c)  accuracy,  which  determines  if  what 
is  sensed  is  accurately  classified  as  friend  or  foe  or  neutral;  and  (d)  reliability,  which  is  the 
likelihood  that  a  message  gets  through. 

To  summarize,  four  key  factors  were  varied  in  the  experiments: 

•  Reliability  (0-100  percent) 

•  Capacity  (0-20  messages/step) 

•  Latency  (0-4  time  steps) 

•  Accuracy  (50-100  percent) 

Data  Farming  Results 

The  synthesis  of  a  meta-model  of  warfighter  effectiveness  for  this  experiment  is  described  in 
the  remaining  paragraphs  of  this  section. 

Table  3.1  describes  the  experimental  design.  The  meta-model  that  was  synthesized, 
as  described  above,  used  the  data  from  the  results  of  the  experiments.  Linear  least  squares 
(LLS)  regression  was  employed.  After  a  number  of  attempts,  a  reasonable  fit  was  achieved  as 
evidenced  by  the  R-square  value  of  0.85  and  a  significance  level  that  was  less  than  0.001  (a 
plot  of  actual  data  versus  model  [predicted]  data  is  in  Appendix  A).  It  was  found  that  using 
the  natural  log  (Ln)  of  capacity  and  capacity  as  terms  improves  the  fit.4 


2  To  distinguish  between  the  two,  MANA  refers  to  intra-squad  SA  as  organic  and  to  inter-squad  SA  as  inorganic. 

3  Loss  ratio  is  the  number  of  Red  casualties  divided  by  the  number  of  Blue  casualties. 

4  The  best  fit  treated  capacity  and  the  log  of  capacity  as  separate  terms,  but  for  the  purpose  of  illustrating  the  marginal  im¬ 
pact  of  capacity  as  a  parameter,  we  combined  them  in  a  different  model. 
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Table  3.1 

A  Full  Factorial  Set  of  Experiments  (First  Set) 


Capacity 

Latency 

Accuracy 

Reliability 

Min 

0 

0,2,4 

50%,  100% 

0,25,50,75,100 

5 

0,2,4 

50%,  100% 

0,25,50,75,100 

10 

0,2,4 

50%,  100% 

0,25,50,75,100 

15 

0,2,4 

50%,  100% 

0,25,50,75,100 

Max 

20 

0,2,4 

50%,  100% 

0,25,50,75,100 

A  profile  of  the  impact  of  the  factors  for  a  similar  model  is  shown  in  Figure  3.3  and 
illustrates  the  role  of  the  terms  capacity  and  log  of  capacity.  The  positive  slopes  for  the  reli¬ 
ability  factor  and  the  capacity  factor  follow  intuition,  i.e.,  the  more  reliable  the  message 
transmission,  the  more  effective  the  warfighter  in  terms  of  loss  ratio  (LR).5  Similarly,  an  in¬ 
crease  in  message  latency  hurts  effectiveness.  This  is  intuitive  as  well.  Both  Figure  3.3  and 
Table  3.2  indicate  how  reliability  is  significant  relative  to  the  other  factors. 

Figure  3.3 

Profile  of  Impact  of  Various  Factors  for  Similar  Model 


The  model  is  the  closed-form  expression  that  is  shown  below  (with  rounded  num¬ 
bers)  and  in  Appendix  A.  In  the  presentation  of  this  equation  and  others  that  follow,  the 
mean  of  the  factor  is  subtracted  from  each  factor  (e.g.,  “Capacity  -  10”). 

LR  =  (-13.25)  +  0.037  *  Reliability  -  1.149  *  Capacity  +  (Reliability  -  50)  *  ((Capacity  -  10)  * 
-0.0027)  -  0.204  *  Latency  +  (Reliability  -  50)  *  ((Latency  -  2)  *  -0.0044)  +  (Capacity  -  10)  * 
((Latency  -  2)  *  0.015)  +  0.015  *  Accuracy  +  (Reliability  -  50)  *  ((Accuracy  -  75)  *  0.000107) 
+  (Capacity  -  10)  *  ((Accuracy  -  75)  *  -0.00072)  +  (Latency  -  2)  *  ((Accuracy  -  75)  * 
-0.0044)  +  12.073  *  (ln(Capacity+2))  +  (Reliability  -  50)  *  (((Ln(Capacity+2))  -  2.21)  * 
0.041)  +  (  Capacity  -  10)  *  (((Ln(Capacity+2))  -  2.21)  *  0.46657)  +  (Latency  -  2)  * 
(((Ln(Capacity+2))  -  2.21)  *  -0.211)  +  (  Accuracy  -  75)  *  (((Ln(Capacity  +  2))  -  2.21)  * 
0.013). 


^  The  Loss  ratio  (LR)  is  a  typical  measure  of  warfighter  and  force  effectiveness.  It  is  defined  here  as  follows:  LR  =  Red  casu¬ 
alties  /  Blue  casualties. 
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Table  3.2 

Significance  of  Terms 


Term 

Estimate 

t  Ratio 

Prob>|t| 

Intercept 

-13.25166 

-3.23 

0.0014 

Reliability 

0.0370058 

26.21 

<  .0001 

Capacity 

-1.149098 

-4.03 

<  .0001 

(Reliability  -  50)*(Capacity  -  10) 

-0.002738 

-4.46 

<  .0001 

Latency 

-0.204361 

-6.69 

<  .0001 

(Reliability  -  50)*(Latency  -  2) 

-0.004444 

-5.14 

<  .0001 

(Capacity  -  1 0)*(Latency  -  2) 

0.0148075 

1.11 

0.2668 

Accuracy 

0.014568 

7.30 

<  .0001 

(Reliability  -  50)*(Accuracy  -  75) 

0.0001072 

1.90 

0.0587 

(Capacity  -  1 0)*(Accuracy  -  75) 

-0.000716 

-0.82 

0.4107 

(Latency  -  2)*(Accuracy  -  75) 

-0.004432 

-3.63 

0.0003 

Ln(Capacity  +  2) 

12.072673 

4.34 

<  .0001 

(Reliability  -  50)*(Ln(Capacity  +  2)-2. 20964) 

0.0407276 

7.96 

<  .0001 

(Capacity  -  10)*(Ln(Capacity  +  2)  -2.20964) 

0.4665696 

3.28 

0.0012 

(Latency  -  2)*(Ln(Capacity  +  2)  -2.20964) 

-0.211188 

-1.91 

0.0575 

(Accuracy  -  75)*(Ln(Capacity  +  2)  -2.20964) 

0.0126874 

1.75 

0.0805 

Table  3.2  shows  how  reliability  is  significant  relative  to  the  other  factors.  It  suggests 
that  the  majority  of  the  terms  in  the  model  are  significant,  assuming  that  likelihoods  (see  the 
third  column)  less  than  5  percent  are  significant.  Notable  exceptions  include  the  cross- 
product  terms  Capacity* Accuracy  and  Capacity*Latency. 

Using  the  model  above,  plots  of  performance  (as  indicated  by  the  LR)  can  be  gener¬ 
ated  for  various  parameters.  Figure  3.4  shows  how  the  performance  varies  as  communication 
capacity  is  varied.  With  LR  as  a  surrogate  for  warfighter  effectiveness  and  communications 
capacity  a  surrogate  for  networking  capability,  we  argue  that  increases  in  warfighter  effective¬ 
ness  resulting  from  increased  networking  capability  eventually  diminish.  Further  detail  on 
selected  simulation  runs  from  this  experiment  is  found  in  Appendix  A. 

Observations 

One  clear  conclusion  can  be  drawn  from  Figure  3.4:  Warfighter  effectiveness  was  affected  by 
communication  capacity.  Specifically,  effectiveness,  as  measured  by  LR,  could  be  cut  in  half 
without  sufficient  capacity;  message  latency  (delay)  affected  warfighter  effectiveness  by  as 
much  as  50  percent  for  a  given  capacity.  The  experiments  and  model  above  assumed  that 
communication  characteristics  were  static/fixed,  i.e.,  capacity  remained  constant  during  the 
simulation,  as  did  reliability  and  the  other  parameters.  These  results  have  to  be  considered 
specific  to  the  scenario. 

As  shown  earlier,  communication  capacity  and  reliability  are  variable  and  should  be 
modeled  dynamically.  MANA  was  modified  to  insert  a  dynamic  model.  Other  factors,  like 
sensing  and  the  rate  that  targets  can  be  prosecuted,  were  also  added.  The  results  are  discussed 
in  the  next  section. 
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Figure  3.4 

Plot  of  Warfighter  Effectiveness  (LR)  as  Capacity  Varies 
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Second  Set  of  Experiments:  Using  a  Dynamic  Communication  Model 

Description  of  Scenario 

In  this  scenario,  as  in  experiment  1 ,  a  squad  of  six  Blue  infantry  armed  with  direct-fire  weap¬ 
ons  is  moving  from  a  position  in  the  northwest  to  the  southeast  corner.  The  soldiers  are  to 
minimize  enemy  contact  but  engage  if  necessary,  and  based  on  that  contact  find  the  best  way 
to  reach  the  waypoint  in  the  southeast.  They  have  the  support  of  an  indirect-fire  weapon 
with  enough  range  to  cover  the  entire  city  but  are  outmatched  (100  to  7)  in  terms  of  sheer 
numbers.  If  the  Blue  force  is  conservative  in  its  approach,  it  is  able  to  reach  the  objective. 
The  dynamic  reliability  model  hurts  communication  capabilities  but  provides  a  more  realistic 
reliability  in  terms  of  the  impact  of  congestion. 

Overview  of  Data  Farming  Effort 

In  this  experiment,  reliability  is  not  a  factor  included  in  the  data  farming  effort.  Instead,  as  is 
the  case  for  the  remaining  experiments,  a  meta-model  of  communication  reliability  was  in¬ 
serted  directly  into  MANA.  We  chose  a  meta-model  that  had  a  quadratic  form,  i.e.,  the 
equation  we  chose  to  use  to  characterize  communication  reliability  is  of  the  form  ax 2  +  bx  + 
c,  where  x  =  load  on  the  network  as  a  fraction  of  capacity.6  This  was  based  on  experimental 
simulations  documented  in  Porche,  Jamison,  and  Herbert  (2004)  using  the  Qualnet  simula¬ 
tor.  A  plot  of  the  actual  equation  used  is  shown  in  Figure  3.5.  MANA  was  modified  to  utilize 
this  equation  instead  of  a  static  setting.  Thus,  a  dynamic  communication  meta-model  (of 
reliability)  allows  MANA  to  vary  reliability  based  on  message  traffic  and  channel  capacity 
during  each  time  step.  The  dynamic  reliability  model  appropriately  reduces  communication 
capabilities  but  provides  more  realistic  reliability  in  terms  of  the  impact  of  congestion. 


6  The  values  we  estimated  for  the  coefficients  are  a  =  -4.456,  b  =  -0.875,  and  c  =  1.0145. 
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Figure  3.5 

Reliability  Decreases  as  the  Ratio  of  Data  Rate  to  Capacity  Increases 


Reliability  Decreases  as  the  Ratio  of  Data  Rate  to 
Capacity  Increases 


-4.456x2-0.8749x+  1.0145 


Design  of  Experiments 

Figure  3.6  shows  the  parameters  chosen  to  address  NCO  components  during  the  simulations 
run  as  outlined  in  Table  3.3.  There  were  five  levels  for  capacity,  three  levels  for  latency,  three 
levels  for  accuracy,  four  levels  for  targeting  rates,  and  four  levels  for  sensing.  This  accounts 
for  720  excursions.  Each  (stochastic)  design  point  is  run  50  times,  so  a  total  of  36,000  runs 
were  made  for  all  of  the  values  in  Table  3.3. 

Figure  3.6 

The  Factors  Chosen  Address  the  Components  of  NCO 


.  Targeting: 
Accuracy 
Target  Rate 


Sensing 

Sensing 


Ccmmunicaticn 

Capacity 

Latency 

Reliability 
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Table  3.3 

A  Full  Factorial  Set  of  Experiments  (Second  Set) 


Capacity 

Latency 

Accuracy 

Targeting  Capability 
(Target  Rate) 

Sensing 

Min 

20 

0,1,2 

50%,  75%,  100% 

50%,  100%,  150%,  200% 

25,50,75,100 

40 

0,1,2 

50%,  75%,  100% 

50%,  100%,  150%,  200% 

25,50,75,100 

60 

0,1,2 

50%,  75%,  100% 

50%,  100%,  150%,  200% 

25,50,75,100 

80 

0,1,2 

50%,  75%,  100% 

50%,  100%,  150%,  200% 

25,50,75,100 

Max 

100 

0,1,2 

50%,  75%,  100% 

50%,  100%,  150%,  200% 

25,50,75,100 

Data  Farming  Results 

Using  the  same  methods  as  in  the  first  set  of  experiments,  another  meta-model  was  synthe¬ 
sized  using  the  data  from  the  experiments  above.  Linear  least  squares  regression  was  em¬ 
ployed.  A  reasonable  fit  (adjusted  r2  =  0.895,  the  significance  is  less  than  0.001)  was 
achieved.  A  plot  of  actual  data  versus  predicted  data  is  shown  in  Appendix  A. 

Analysis  of  this  model  highlights  how  sensing  and  communication  factors  (capacity 
and  latency)  significantly  affect  warfighter  effectiveness  as  measured  by  the  loss  ratio  (LR). 
Target  rate  and  accuracy  were  less  of  a  factor  for  this  scenario. 


Figure  3.7 

Profile  of  the  Impact  of  the  Factors  on  Warfighter  Effectiveness 


The  significance  of  the  model  terms  is  shown  in  column  three  of  Table  3.4.  The  lack 
of  significance  of  target  rate,  by  itself,  is  apparent;  the  likelihood  that  its  coefficient  is  actually 
zero  is  approximately  40  percent. 

Contour  plots  of  the  raw  data  from  the  simulation  runs  depict  the  relative  sensitivi¬ 
ties  of  the  sensing  and  communication  capacities.  They  are  shown  in  Figure  3.8. 
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Figure  3.8 

Relative  Impact  of  Communication  Capacity  and  Latency  and  Sensing  Ability  on  Warfighter 
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Table  3.4 

Model  Formula  with  Tests  of  Terms 


Term 

Estimate 

t  Ratio 

Prob  >  |t| 

Intercept 

-72.620518 

-3.585856 

0.000363804 

Capacity 

-0.483965 

-3.464287 

0.000570206 

Latency 

-2.048534 

-35.827286 

0.000000000 

(Capacity  -  56.31 58)*(Latency  -  0.94737) 

-0.006666 

-0.819997 

0.412549071 

Accuracy 

0.036934 

16.300223 

0.000000000 

(Latency  -  0.94737)*(Accuracy  -  75) 

-0.004117 

-1.501654 

0.133721348 

TargetRate 

0.000699 

0.834888 

0.404118945 

(Latency  -  0.94737)*(TargetRate  -  125) 

-0.005131 

-5.056072 

0.000000572 

Sensing 

0.067194 

40.108477 

0.000000000 

(Capacity  -  56.31 58)*(Sensing  -  62.5) 

-0.000842 

-3.545658 

0.000422694 

(Latency  -  0.94737)*(Sensing  -  62.5) 

-0.024150 

-11.880994 

0.000000000 

(Accuracy  -  75)*(Sensing  -  62.5) 

0.000358 

4.419294 

0.000011786 

(TargetRate  -  125)*(Sensing  -  62.5) 

0.000104 

3.467495 

0.000563580 

Ln(capacity) 

26.801712 

3.846034 

0.000133109 

(Capacity  -  56.31 58)*(Ln(capacity)  -  3.87424) 

0.193605 

2.864815 

0.004321080 

(Latency  -  0.94737)*(Ln(capacity)  -  3.87424) 

1.554273 

3.816312 

0.000149760 

(Accuracy  -  75)*(Ln(capacity)  -  3.87424) 

-0.018908 

-4.854958 

0.000001  544 

(Sensing  -  62.5)*(Ln(capacity)  -  3.87424) 

0.075135 

6.311532 

0.000000001 
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Observations 

Sensing  capability  seems  critical  from  our  experiments;  perhaps  communications  capacity  is 
also  as  critical.  The  most  important  observation  is  that  all  three  trade  off  against  each  other 
to  some  extent.  Increasing  just  one  of  the  three  capabilities,  without  adjusting  the  others, 
may  not  result  in  a  gain  to  warfighter  effectiveness.  In  other  words,  for  this  scenario,  assess¬ 
ments  of  the  impacts  of  communication  capacity  without  sufficient  consideration  of  sensing 
capability  could  be  misleading,  and  vice-versa.  As  shown  in  Figure  3.9,  without  good  accu¬ 
racy  and  targeting  capability,  results  are  very  poor.  The  chart  shows  what  we  believe  is  intui¬ 
tive:  LR  is  proportional  to  Accuracy  *  TargetRate. 

Figure  3.9 

Accuracy  Versus  Target  Rate 


Third  Set  of  Experiments:  Expanded  Three-Squad  Scenario 

Description  of  Expanded  Scenario 

To  better  understand  the  marginal  impact  of  communications,  an  expanded  scenario  was 
developed.  A  second  squad  of  six  infantry  was  added  to  the  initial  scenario.  Each  squad  had 
its  own  communications  link  to  pass  messages  to  the  indirect-fire  squad.  As  shown  in  Figure 
3.10,  each  squad’s  link  is  independent  of  the  others.  The  existing  numbers  of  Red  forces 
were  doubled  in  an  attempt  to  generate  increased  communications  activity. 

At  the  end  of  each  time  step,  a  random  order  is  determined  in  processing  the  com¬ 
munication  queues7  related  to  each  squad.  The  two  squads  do  not  share  capacity;  capacity  is 
preallocated  to  each  one. 


7  The  communication  queue  is  the  location  where  all  messages  from  the  squad  are  placed  before  being  sent  on  the  commu¬ 
nications  link.  This  is  where  a  message  is  sent,  or  not  sent,  based  on  reliability  and  capacity  (among  other  factors). 
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Figure  3.10 

Blue  Force  Communication  Link  (Expanded  Scenario) 


Data  Farming  Results 

The  design  of  experiments  is  as  follows:  216  combinations  of  factors  were  considered  as 
shown  in  Table  3.5.  Fifty  runs  were  completed  for  each  combination  of  factors  for  a  total  of 
10,800  runs. 

Table  3.5 

A  Full  Factorial  Set  of  Experiments  Totaled  10,800  Runs 


Squad  1 
Capacity 

Squad  2 
Capacity 

Latency 

Accuracy 

Targeting 

Capability 

Sensing 

Min 

10 

10,40,80 

0,4 

50%, 100% 

50%,  1 00%, 250% 

50,100 

40 

10,40,80 

0,4 

50%,  100% 

50%,  1 00%, 250% 

50,100 

Max 

80 

10,40,80 

0,4 

50%, 100% 

50%,  1 00%, 250% 

50,100 

A  model  was  successfully  fit  to  the  resulting  data.  A  better  fit  was  achieved  using  the 
natural  log  of  LR  rather  than  simply  LR.  This  was  noticed  for  all  subsequent  models.  The  fit 
has  an  adjusted  R-square  value  of  0.94  and  a  significance  of  less  than  0.0001.  (A  plot  of  ac¬ 
tual  versus  model-predicted  data  as  well  as  the  model’s  equation  can  be  found  in  Appendix 
A).  Analysis  of  the  significance  of  each  term  is  shown  in  Table  3.6. 
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Table  3.6 

Terms  of  Model  from  Third  Set  of  Experiments 


Term 

Estimate 

t  Ratio 

Prob>|t| 

Intercept 

-1.296511531 

-2.07077503 

0.03922918 

CapacitySqdl 

0.023468764 

4.82209667 

2.25555E-06 

CapacitySqd2 

0.027399798 

5.62980117 

4.1246E-08 

(CapacitySqdl  -  43.3333)*(CapacitySqd2  -  43.3333) 

-0.0002843 

-1.67501117 

0.094967144 

Latency 

-0.990199934 

-14.1907054 

2.25771  E-35 

(CapacitySqdl  -  43.3333)*(Latency  -  2) 

0.006961224 

2.860627372 

0.004522931 

(CapacitySqd2  -  43.3333)*(Latency  -  2) 

0.005752684 

2.363993223 

0.018711918 

Accuracy 

0.042456186 

7.605575391 

3.62235E-13 

(CapacitySqdl  -  43.3333)*(Accuracy  -  75) 

-0.00021383 

-1.09838266 

0.272912341 

(CapacitySqd2  -  43.3333)*(Accuracy  -  75) 

-9.84688E-05 

-0.50580626 

0.613361652 

(Latency  -  2)*(Accuracy  -  75) 

-0.003749573 

-1.34339245 

0.180153507 

Targeting 

0.002041734 

1.243327757 

0.214711415 

(CapacitySqdl  -  43.3333)*(Targeting  -  133.333) 

-1.68459E-05 

-0.29415518 

0.768841403 

(CapacitySqd2  -  43.3333)*(Targeting  -  133.333) 

-8.33756E-06 

-0.14558622 

0.884345165 

(Latency  -  2)*(Targeting  -  133.333) 

-0.002092867 

-2.54893154 

0.01129988 

(Accuracy  -  75)*(Targeting  -  133.333) 

1 .1 21 69E-05 

0.170765478 

0.864522526 

Sensing 

0.116404973 

30.71010418 

6.70038E-95 

(CapacitySqdl  -  43.3333)*(Sensing  -  53.3333) 

0.000394817 

2.986756339 

0.003050541 

(CapacitySqd2  -  43.3333)*(Sensing  -  53.3333) 

0.000491893 

3.721127598 

0.000236485 

(Latency  -  2)*(Sensing  -  53.3333) 

-0.019494371 

-10.2860582 

1.79309E-21 

(Accuracy  -  75)*(Sensing  -  53.3333) 

0.000738129 

4.868354441 

1.8171  IE-06 

(Targeting  -  133.333)*(Sensing  -  53.3333) 

4.31927E-05 

0.968402668 

0.333618207 

Observations:  Quantifying  the  Relative  Impact  of  Sensors  and  Cognition 

It  was  observed  that  sensing  is  very  significant.  Table  3.6  (for  experiment  3)  quantifies  the 
significance  of  this  term.  It  should  be  noted  that  when  communication  capability  (capacity 
and  latency)  is  good,  sensing  has  a  large  impact  on  warfighter  effectiveness.  If  communica¬ 
tion  capability  is  poor  (e.g.,  increased  latency  coupled  with  low  capacities  for  both  squads), 
then  the  sensors  are  ineffective.  This  is  what  is  evident  from  Figure  3.11,  which  indicates  the 
approximate  change  in  LR  for  a  small  change  in  one  of  the  factors.  This  also  follows  intui¬ 
tion.  A  common  thread  between  this  experiment  and  previous  experiments  was  that  message 
latency  again  proved  it  could  impact  LRs  by  as  much  as  50  percent.  Clearly,  the  impact  of 
communication  factors  cannot  be  ignored. 
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Figure  3.11 

Profile  of  Factors  for  Experiment  3 


Fourth  Set  of  Experiments:  "Mosque-UAV"  Scenario  Description 

Two  variants  of  a  mosque-scenario  are  detailed  as  follows. 

Direct-Fire  Case  (Variant  1) 

Description  of  Scenario.  The  scenario  used  is  depicted  in  Figure  3.12  and  is  outlined 
as  follows.  Blue  forces  are  attempting  to  secure  an  area  around  a  mosque  and  stop  Red  forces 
from  entering.  Blue  relies  on  a  set  of  four  12-man  infantry  squads.  Blue  force  has  UAVs  that 
help  with  sensing  and  communicating.  The  squads  have  direct-fire  weapons  only.  Red’s  mis¬ 
sion:  seek  safe  haven  in  the  mosque  where  Blue  cannot  enter.  Blue  is  seeking  enemy  contact 
because  their  goal  is  to  stop  Red  from  reaching  the  mosque.  To  keep  Blue  from  firing  on  tar¬ 
gets  inside  the  mosque,  a  special  terrain  type  was  added  that  gave  the  Red  forces  100  percent 
concealment  while  inside.  There  are  two  squads  of  Red  forces,  six  leaders,  and  thirty  security 


Figure  3.12 

UAVs  in  Urban  Area:  Direct-Fire  Only 
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forces.  The  security  forces  are  better  equipped  than  their  counterparts  in  experiments  1-3, 
and  will  stay  with  the  leaders  at  all  costs.  They  are  successful  when  the  leaders  are  able  to 
move  unimpeded  to  the  mosque. 

The  result:  It  was  observed  that  the  more  far-ranging  the  sensing  ability  of  the  Blue 
force  infantry  squads,  the  worse  the  outcome.  The  reason:  the  fight  between  Red  and  Blue 
infantry,  in  this  scenario,  was  mostly  even.  So,  increased  sensing  increased  engagements  over¬ 
all  and  thus  caused  casualties  on  both  sides.  For  example,  the  maximum  LR  seen  for  this  sce¬ 
nario  was  one  where  the  sensing  capability  of  the  Blue  squads  was  most  limited. 

Results  of  Data  Farming  Effort.  A  model  of  the  log  of  the  LR  was  fit  from  the  out¬ 
put  of  the  simulation  experiments  run  in  MANA.  A  reasonable  fit  was  achieved  (R-square 
value  is  0.84).  The  factors8  involved  were  varied  as  shown  in  Table  3.7  and  are  explained  as 
follows: 

•  “RangerSensing”  and  “UAVSensing”  represent  the  ability  of  the  infantry  squads  and 
the  UAVs  to  sense  the  presence  of  Red  forces. 

•  “Accuracy”  determines  whether  what  is  sensed  is  accurately  classified  as  friend  or  foe 
or  neutral. 

•  “Latency”  represents  any  delay  in  terms  of  the  message  that  gets  passed  when  some¬ 
thing  is  sensed  and  this  information  is  communicated  out. 

•  “Capacity”  reflects  the  number  of  messages  that  can  be  transmitted  at  one  period. 

Analysis  of  the  factors  shows  the  ability  of  the  infantry  (“Ranger”)  squads  to  sense  Red 
forces  (labeled  “RangerSensing”)  predominated;  increasing  the  communication  capacity  or 
sensing  ability  (RangerSensing  or  UAVSensing)  did  not  significantly  increase  the  warfighter 
effectiveness  as  measured  by  the  LR. 

Appendix  A  contains  the  model’s  formula.  Table  3.8  confirms  the  significance  of  the 
sensing  ability  (RangerSensing)  on  the  Blue  force’s  performance  and  this  is  seen  in  Figure 
3.13.  Figure  3.14  incorporates  the  model’s  formula  and  shows  a  decrease  in  warfighter  per¬ 
formance,  as  measured  by  the  LR,  when  RangerSensing  is  increased  and  the  other  factors, 
which  have  relatively  little  effect  according  to  Figure  3.13,  are  held  fixed.9 


Table  3.7 

Design  of  Experiments 


Ranger 

UAV 

Capacity 

Sensing 

Sensing 

Accuracy 

Latency 

Min 

5 

10,30,50 

50,100,150 

50%,  100% 

0,4 

25 

10,30,50 

50,100,150 

50%,  100% 

0,4 

50 

10,30,50 

50,100,150 

50%,  100% 

0,4 

75 

10,30,50 

50,100,150 

50%,  100% 

0,4 

Max 

100 

10,30,50 

50,100,150 

50%,  100% 

0,4 

8  Reliability  is  modeled  dynamically. 

9  “UAVSensing”  was  fixed  at  150,  “Accuracy”  was  75,  “Latency”  was  0,  and  “Capacity”  was  148. 
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Table  3.8 

Terms  of  Model:  Direct-Fire  Case  (Variant  1) 


Term 

Estimate 

t  Ratio 

Prob  >  |t| 

Intercept 

-2.6859045 

-37.6396 

1.5E-108 

RangerSensing 

0.0002678 

0.354185 

0.7234805 

UAVSensing 

0.0015678 

6.348075 

9.364E-10 

(RangerSensing  -  30)*(UAVSensing  -  100) 

-3.967E-05 

-2.6227 

0.0092267 

Accuracy 

0.0282132 

57.1183 

2.43E-1 51 

(RangerSensing  -  30)*(Accuracy  -  75) 

-4.98E-05 

-1 .64642 

0.1008579 

(UAVSensing  -  100)*(Accuracy  -  75) 

1.397E-05 

1.414376 

0.158421 

Latency 

-0.1249862 

-20.243 

8.747E-56 

(RangerSensing  -  30)*(Latency  -  2) 

-0.0005452 

-1.44194 

0.1504954 

(UAVSensing  -  1 00)*(Latency  -  2) 

-0.0004222 

-3.4192 

0.0007266 

(Accuracy  -  75)*(Latency  -  2) 

-0.0033407 

-13.5266 

4.429E-32 

IFRange 

0.1010521 

13.36328 

1.648E-31 

(RangerSensing  -  30)*(IFRange  -  4) 

0.0008309 

1.794275 

0.0739047 

(UAVSensing  -  100)*(IFRange  -  4) 

0.0001704 

1.126834 

0.2608282 

(Accuracy  -  75)*(IFRange  -  4) 

0.0026394 

8.726017 

2.948E-16 

(Latency  -  2)*(IFRange  -  4) 

-0.0206377 

-5.45831 

1.101E-07 

Ln(capacity) 

0.2115579 

19.03393 

1.45E-51 

(RangerSensing  -  30)*(Ln(Capacity)  -  3.3364) 

0.0007487 

1.099934 

0.2723555 

(UAVSensing  -  100)*(Ln(Capacity)  -  3.3364) 

0.0011806 

5.311125 

2.303E-07 

(Accuracy  -  75)*(Ln(Capacity)  -  3.3364) 

0.0056576 

12.72535 

2.714E-29 

(Latency  -  2)*(Ln(Capacity)  -  3.3364) 

0.0221507 

3.985814 

8.692E-05 

(IFRange  -  4)*(Ln(Capacity)  -  3.3364) 

0.0263929 

3.877668 

0.000133 

Figure  3.13 

Model  Factors  for  Variant  1 
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Figure  3.14 

Improved  Sensing  Resulted  in  Reduced  Performance  in  Variant  1 


MOSQUE-  UAV:  Direct  Fire  Case 


'RangerSensing'  values 


As  noted  earlier,  the  better  Blue’s  sensing,  the  worse  its  outcome.  Additional  excur¬ 
sions  that  varied  the  squad  size  showed  no  difference  in  outcome  in  terms  of  average  LR. 
These  are  shown  in  Table  3.9.  The  reason:  The  direct-fire  fight  is  still  fairly  even  in  terms  of 
capabilities.  Increased  sensing  increases  casualties  somewhat  evenly  on  both  sides.  In  the  next 
section,  this  scenario  is  modified  to  enhance  the  Blue  force’s  makeup  by  adding  indirect-fire 
capabilities. 


Table  3.9 

Variant  1  With  Squad  Size  Varied 


6-man  squads 

9-man  squads 

12-man  squads 

Mean  LR 

1.64 

1.67 

1.76 

Max  LR 

3.75 

3.83 

4.18 

Min  LR 

0.7 

0.66 

0.78 

Indirect-Fire  Case  (Variant  2):  A  Scenario  With  Enhancements  to  Blue 

Description  of  the  Scenario.  We  modified  the  scenario  by  adding  an  indirect-fire 
weapon  to  the  Blue  force,  which  also  has  four  UAVs  that  provide  sensing  and  communicat¬ 
ing  capability  (see  Figure  3.15).  We  refer  to  this  as  Variant  2.  The  UAVs  are  spaced  evenly 
and  move  in  a  preplotted  circular  fashion  above  the  city.  They  are  fully  networked,  as  shown 
in  Figure  3.16.  The  Red  forces  will  occasionally  shoot  at  them,  but  they  are  difficult  to  hit. 
Blue  is  seeking  enemy  contact  because  its  goal  is  to  stop  Red  from  reaching  the  mosque.  To 
keep  Blue  from  firing  on  targets  inside  the  mosque,  a  special  terrain  type  was  added  that  gave 
the  Red  forces  100  percent  concealment  while  inside.  The  security  forces  are  better  equipped 
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than  their  counterparts  in  experiments  1-3,  and  will  stay  with  the  leaders  at  all  costs.  They 
are  successful  when  the  leaders  are  able  to  move  unimpeded  to  the  mosque. 

Figure  3.15 

UAVs  in  Urban  Scenario  with  Indirect-Fire  Support  (Variant  2) 


Figure  3.16 

Situational  Awareness  Is  Shared  Throughout  All  Blue  Forces 
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Results  of  Data  Farming  Effort.  The  results  from  augmenting  the  Blue  force  are  sig¬ 
nificant:  Mean  LR  was  2.2,  with  maximum  LR  an  order  of  magnitude  higher.  The  UAVs  in 
the  scenario  are  fully  networked,  so  SA  is  shared  among  all  Blue  forces.  In  all,  288  combina¬ 
tions  of  factors  resulted  in  14,400  runs.  A  model  of  the  log  of  the  LR  was  fit  from  the  output 
of  the  simulation  experiments  run  in  MANA,  achieving  an  excellent  fit  (r2  value  of  0.95). 
The  significant  terms  of  the  model  are  shown  in  Table  3.10. 

Analysis  of  the  marginal  impact  of  the  individual  factors  is  shown  in  Figure  3.17.  Ac¬ 
curacy  is  the  predominant  factor:  The  more  accurate  the  SA  messages,  the  more  effective  the 
Blue  force  is  in  terms  of  LR.  Increased  accuracy  also  decreased  casualties  to  noncombatants. 
Increased  sensing  on  the  UAV  and  improved  communicating  ability  now  help  in  this  sce¬ 
nario,  unlike  the  previous  one,  where  increased  networking  (or  C4ISR  ability)  hurt  perform¬ 
ance. 

It  also  shows  that  unlike  the  earlier  scenario  runs  that  did  not  have  an  indirect-fire 
weapon,  increasing  the  communication  capacity  did  improve  warfighter  effectiveness  as 
measured  by  the  LR.  Overall,  the  implication  in  comparing  results  from  the  two  scenarios  is 
that  network  capability  may  sometimes  increase  warfighter  effectiveness,  and  sometimes  it 
may  not.  The  outcome  is  dependent  on  the  scenario  and  the  force  makeup  involved. 


Figure  3.17 
Profile  of  Factors 
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The  equation  that  represents  the  synthesized  model  was  fit  to  the  data  generated  by 
the  simulation  experiments.  Table  3.10  quantifies  the  significance  of  each  term  of  the  model 
and  confirms  the  significance  of  accuracy. 
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Table  3.10 

Terms  of  Model:  Indirect-Fire  Case  (Variant  2) 


Term 

Estimate 

t  Ratio 

Prob>|t| 

Intercept 

-0.26507268 

-4.54894075 

1. 89561  E-05 

RangerSensing 

-0.00909447 

-13.3391445 

4.90514E-22 

UAVSensing 

0.001383295 

6.212280551 

2.2068E-08 

(RangerSensing  -  30)*(UAVSensing  -  100) 

-3.3933E-05 

-2.48849842 

0.014904617 

Accuracy 

0.002197356 

4.934086155 

4.30331  E-06 

(RangerSensing  -  30)*(Accuracy  -  75) 

-0.00017721 

-6.49787211 

6.42529E-09 

(UAVSensing  -  100)*(Accuracy-  75) 

-8.771  E-06 

-0.98474662 

0.32771747 

Latency 

0.011784977 

2.117020814 

0.037365102 

(RangerSensing  -  30)*(Latency  -  2) 

0.000415976 

1.220249413 

0.225955158 

(UAVSensing  -  1 00)*(Latency  -  2) 

-0.00023843 

-2.14155368 

0.035272111 

(Accuracy  -  75)*(Latency  -  2) 

-0.00021789 

-0.97853726 

0.330758582 

Ln(Capacity) 

-0.02076718 

-2.07233476 

0.041455015 

(RangerSensing  -  30)*(Ln(Capacity)  -  3.33638) 

-0.00058586 

-0.95469111 

0.342609854 

(UAVSensing  -  100)*(Ln(Capacity)  -  3.33638) 

0.000496115 

2.475339985 

0.015423514 

(Accuracy  -  75)*(Ln(Capacity)  -  3.33638) 

0.000556827 

1.389130495 

0.16864782 

(Latency  -  2)*(Ln(Capacity)  -  3.33638) 

-0.01062972 

-2.12145683 

0.036978858 

Other  Performance  Measures 

This  scenario  was  used  again  with  more  simulation  runs.  However,  instead  of  measuring  the 
outcome  with  LR,  the  outcome  was  redefined  as  a  binary  variable  called  “mission  success.” 
Specifically,  Blue  succeeds  if  it  stops  all  Red  high-value  targets  from  reaching  the  mosque  in 
the  center  of  town.  If  one  of  the  targets  enters  the  mosque  successfully,  then  Blue  will  have 
failed  its  mission.  The  mean  success  rate  of  all  excursions  is  listed  in  the  tables  that  follow. 

As  the  data  in  Table  3.1 1  suggests,  changing  the  size  of  the  squads  has  a  positive  im¬ 
pact  on  the  success  rate  of  the  Blue  force  for  the  scenario  where  Blue  only  has  direct-fire 
weapons,  despite  the  small  change  in  LR.  Of  course,  the  results  from  the  indirect-fire  ex¬ 
periment,  where  Blue  now  has  indirect-fire  capabilities  added,  show  it  to  be  much  more  ef¬ 
fective.  Interestingly,  Blue  doesn’t  benefit  from  adding  more  men  per  squad  for  this  indirect- 
fire  case;  the  data  in  Table  3.12  suggests  that  the  likelihood  of  success  remains  virtually  un¬ 
changed  as  a  function  of  squad  size. 

Table  3.11 

Direct-Fire  (Variant  1)  Excursions  with  Binary  Mission  Success  Variable 


6-man 

9-man 

12-man 

Mean  Loss  Ratio 

1.64 

1.67 

1.76 

Max  Loss  Ratio 

3.75 

3.83 

4.18 

Min  Loss  Ratio 

0.70 

0.66 

0.78 

Likelihood  Blue  Objective  Achieved 

10% 

17% 

28% 
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Table  3.12 

Indirect-Fire  (Variant  2)  Excursions  with  Binary  Mission  Success  Variable 


6-man 

9-man 

12-man 

Mean  loss  ratio 

1.92 

1.19 

1.19 

Max  loss  ratio 

5.72 

3.65 

4.13 

Min  loss  ratio 

0.65 

0.52 

0.48 

Likelihood  Blue  Objective  Achieved 

34% 

39% 

38% 

A  new  model  was  fit  using  the  logit  function  (see  Appendix  A)  so  that  the  perform¬ 
ance  metric  is  now  the  likelihood  of  mission  success.  The  figure  below  shows  the  relative  im¬ 
pact  of  factors  against  Blue’s  mission  success  for  the  12-man  squad.  The  accuracy  of  informa¬ 
tion  coming  in  from  the  battlefield  has  significant  impact  on  the  success  of  the  Blue  team.  If 
high-value  targets  can  be  identified  correctly,  the  probability  for  success  increases  by  as  much 
as  20  percent. 

Using  the  model  described  earlier,  we  see  the  impacts  of  accuracy  in  context.  Force 
makeup  is  important  both  in  composition  (indirect-fire  or  none)  and  size  (12-man  versus  6- 
man).  Comparisons  of  Tables  3.11  and  3.12  show  how  the  use  of  precision  indirect-fire  fa¬ 
cilitates  a  nearly  equivalent  measure  of  mission  success  with  far  less  manpower  (e.g.,  squad 
size).  This  is  consistent  with  some  of  the  hypotheses  of  NCO.  This  is  made  even  clearer  in 
Figures  3.18  and  3.19. 

Figure  3.18 

The  Communication  of  Accurate  Information  Is  Needed  to  Increase  Odds  of  Success  (p) 
in  Variant  1 


Accurate  Reporting  of  High  Value  Target  Locations  Combined 
With  Good  Communications  Can  Boost  Mission  Success  Rates 
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Figure  3.19 

Force  Makeup  Was  a  Key  Determinant 


Accurate  Asessment  of  High  Value  Targets  Improves 
Probability  of  Mission  Success 


Accuracy 


Observations 

The  implication  in  comparing  results  from  the  scenarios  just  described  is  that  network  capa¬ 
bility  may  sometimes  increase  warfighter  effectiveness.  Clearly,  there  is  a  minimum  force 
makeup  required  before  any  enhancement  can  be  seen.  The  curve  that  represents  warfighter 
effectiveness  given  varying  levels  of  networking  capability  can  be  nonincreasing  or  nonde¬ 
creasing  depending  on  the  makeup  of  the  force  and  firepower.  Downes,  Kwinn,  and  Brown 
(2004)  had  similar  results:  With  regard  to  a  direct-fire  only  scenario  and  a  simulation  of  Blue 
infantry  versus  Red  infantry  in  a  small  urban  setting,  they  noted  that  they  could  not  con¬ 
clude  that  the  increased  situational  awareness  of  the  Land  Warrior-equipped  soldiers  led  to 
increased  lethality  and  survivability;  they  utilized  human-in-the-loop  experimentation  to 
draw  the  conclusion.  For  a  set  of  MANA  simulation  runs  of  infantry  squads  in  an  urban  en¬ 
vironment,  the  same  researchers  again  reached  conclusions  that  were  similar  to  what  this  re¬ 
port  also  found,  i.e.,  “the  force  with  improved  situational  awareness  can  only  take  advantage 
of  their  enhanced  abilities  to  communicate  and  share  a  common  vision  of  the  battlefield  if 
they  are  given  additional  combat  multipliers  such  as  artillery  or  attack  helicopters”  (Downes, 
Kwinn,  and  Brown  2004). 

The  analysis  framework  used  in  this  report  is  one  where  warfighter  effectiveness  is  a 
function  of  networking  parameters  that  are  input  into  and  translated  by  the  MANA  force- 
on-force  simulator.  The  results  presented  suggest  that  no  “level  of  networking”  or  “degree  of 
networking”  can  be  defined  as  adequate,  for  NCO  or  other  concepts,  outside  of  any  context. 
Specifically,  the  structure  of  the  force  and  the  scenario  are  additional  significant  factors.  As 
Figure  3.14  illustrates,  increased  networking  capability  does  not  necessarily  increase  war- 
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fighter  effectiveness.10  As  Figure  3.4  illustrates,  when  there  are  increases,  they  are  not  un¬ 
bounded,  i.e.,  increases  in  warfighter  effectiveness  eventually  diminish  as  a  result  of  increased 
networking  capability. 

Finally,  two  different  performance  metrics  were  considered:  a  loss  ratio  and  a  prob¬ 
ability  of  mission  success.  In  hindsight,  a  combination  of  the  two  metrics  could  be  more  use¬ 
ful.  In  a  generalized  form,  we  can  express  this  as  (1  -  w)LR  +  wP,  where  LR  is  the  loss  ratio, 
P  is  the  probability  of  mission  success,  and  w  is  an  arbitrary  constant  between  zero  and  one 
that  varies  the  importance  of  mission  success  over  loss  ratio.  For  example,  for  w  =  1,  mission 
success  would  be  weighted  such  that  the  performance  measure  is  impacted  largely  by  mission 
success  relative  to  loss  ratio. 

It  is  important  to  keep  these  simulation  results  in  the  proper  context.  Actual  force 
makeup,  network  architecture,  and  communication  performance  may  deviate  from  the  as¬ 
sumptions  in  this  report.  The  authors  would  like  to  claim  that  these  results  remain  applicable 
even  when  finer  resolution  of  technical  and  operational  specifications  are  utilized.  That  re¬ 
mains  to  be  seen.  Fiowever,  the  integration  of  a  high-resolution  communication  network 
simulator  is  under  way  which  may  verify  these  results.  The  tool  being  used  is  Qualnet©. 
Qualnet  (or  other  suitable  high-resolution  networking  modeling  tools)  can  account  for  an¬ 
tenna  models,  routing  protocols,  signal  power,  and  frequency  of  transmission.  Currently,  we 
relied  on  a  meta-model  to  characterize  dynamic  message  reliability.  Although  this  meta¬ 
model  was  synthesized  using  Qualnet,  we  leave  it  to  future  work  to  repeat  these  examinations 
without  the  use  of  meta-models  and  perhaps  with  a  force  makeup  that  reflects  actual  future 
Army  plans  and  official  scenarios.  As  suggested,  this  would  necessitate  the  integration  of 
high-resolution  network  simulators  to  the  force-on-force  tool,  which  may  or  may  not  be 
practical. 


10  We  assume  two  things:  We  can  use  (i)  LR  as  a  surrogate  for  warfighter  effectiveness  and  (ii)  communications  capacity  as 
a  surrogate  for  networking  capability  in  the  case  of  the  sufficient  force  makeup  and  sensing  capacity  as  a  surrogate  for  net¬ 
work  capability.  However,  in  both  cases,  these  factors  dominated  the  other  parameters. 


CHAPTER  FOUR 


Overall  Conclusions 


There  are  many  factors  that  impact  the  effectiveness  of  battlefield  networks.  We  initially 
identified  three  sets  of  factors:  communication,  sensing,  and  cognitive  capabilities.  In  addi¬ 
tion,  we  observed  that  the  force  makeup  affected  the  marginal  impact  of  these  three  factors. 
Some  of  the  critiques  of  the  networked  operation,  as  summarized  in  the  recent  GAO  report, 
are  the  result  of  a  deficiency  in  one  of  these  areas.  For  example,  that  report  (GAO,  2004) 
noted:  “The  improved  ability  to  share  view  of  the  battlefield  and  communicate  quickly  has 
compressed  the  time  required  for  analysis  and  decision  making  such  that  large  increases  in 
the  pace  of  operations  and  the  volume  of  information  have  overwhelmed  commanders  at 
times.”  This  critique  relates  to  improvement  in  communication  without  a  corresponding 
improvement  in  cognitive/fusing  capabilities.  A  second  major  critique  in  the  same  GAO  re¬ 
port  was  as  follows:  “Slow  or  inaccurate  [battle  damage]  assessments  negated  improvements 
in  the  speed  of  operations;  battle  damage  assessments  didn’t  keep  up  with  the  pace  of  opera¬ 
tions.”  This  directly  relates  to  sensing  deficiencies.  Nonetheless,  the  same  report  found  that 
overall  it  was  improvements  in  force  networks  and  the  use  of  precision  weapons  that  were  the 
primary  reasons  for  the  overwhelming  combat  power  in  Operation  Iraqi  Freedom. 

In  this  report,  we  considered  a  few  components  of  the  cost  of  networking  (e.g.,  a 
model  on  the  limits  on  a  communication  channel).  The  model  was  inserted  into  an  agent- 
based,  force-on-force  combat  simulator  for  a  simple  urban  scenario.  One  clear  conclusion  is 
that  warfighter  effectiveness  was  affected  by  communication  capacity.  Specifically,  effective¬ 
ness,  as  measured  by  a  loss  ratio,  could  be  cut  in  half  without  sufficient  capacity;  message  la¬ 
tency  (delay)  affected  warfighter  effectiveness  by  as  much  as  50  percent  for  a  given  capacity. 
We  observed  that  the  improvement  in  effectiveness,  as  a  result  of  increasing  communication 
capacity,  diminishes  eventually.  We  also  observed  a  case  where  warfighter  effectiveness  de¬ 
creases  with  networking  capability. 

Network  capability  may  sometimes  increase  warfighter  effectiveness  and  sometimes  it 
may  not;  it  depends  on  the  scenario  and  the  force  makeup  involved  along  with  the  key  net¬ 
working  parameters  investigated  (sensing,  communicating,  targeting) .  These  observations  are 
counter  to  Metcalfe’s  Law  and  Reed’s  Law.  This  is  illustrated  in  the  notional  curves  of  Figure 
4.1,  which  is  supported  by  data  in  the  body  of  this  report  (see  Figures  3.4  and  3.14)  for  a 
given  scenario  that  had  varied  the  makeup  of  the  force.  Therefore,  we  conclude  that  it  is  not 
only  the  sensing,  communication,  and  certain  cognitive  parameters  of  networking  that  de¬ 
termine  effectiveness,  but  also  the  force  makeup  that  is  utilizing  these  networking  capabili¬ 
ties. 

Although  a  large  space  of  possibilities  was  explored  for  the  parameters  under  consid¬ 
eration,  all  results  need  to  be  taken  in  the  context  of  the  specific  scenarios  simulated.  Specifi¬ 
cally,  the  results  in  this  report  may  be  sensitive  to  the  assumptions  that  are  made  with  regard 
to  network  architecture,  force  makeup,  technology  performance,  and  the  focus  at  the  tactical 
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level.  Nonetheless,  the  results  show  how  agent-based  tools,  when  sufficiently  validated,  can 
be  exploited  to  quantify  the  impact  of  networks  and  networking  performance  to  warfighters. 

Figure  4.1 

Situations  Where  Networking  Capability  Increased  Warfighter  Effectiveness 


It  is  important  to  note  that  communication  capacity  depends  on  frequency  spectrum 
allocation.  Effective  spectral  and  network  management  could  help  improve  available  capac¬ 
ity.  This  will  be  studied  as  part  of  future  work. 


APPENDIX  A 


Synthesized  Models 


First  Set  of  Experiments 

Figure  A.  1  is  a  plot  of  the  actual  versus  predicted  for  the  metamodel  synthesized  for  experi¬ 
ment  1. 


Figure  A.1 

Predicted  Versus  Actual  for  Experiment  1  Model  Fit 


LER  Predicted 

The  equation  for  the  model  resulting  from  analysis  of  the  first  set  of  simulation  runs 
is  shown  below. 


LR  =  (-13.25166)  +  0.037006  *  Reliability -1.14909751293089  *  Capacity  +  (Reliability  -  50)  * 

((Capacity  -  10)  *  -0.0027384)-0.204360923621705  *  Latency  +  (Reliability  -  50)  * 
((Latency -2)  *-0.0044439)  +  (Capacity  -  10)  *  ((Latency -2)  *  0.0148075)+  0.014568  * 
Accuracy  +  (Reliability  -  50)  *  ((Accuracy  -  75)  *  0.00010717)  +  (Capacity  -  10)  * 
((Accuracy- 75)  *-0.00071616)  +  (Latency -2)  *  ((Accuracy -  75)  *-0.0044325)  + 
12.07267  *  (ln(Capacity+2))  +  (Reliability  -  50)  *  (((Ln(Capacity+2))  -  2.20964)  * 
0.040728)  +  (Capacity  -  10)  *  (((Ln(Capacity+2))  -  2.20964)  *  0.46657)  +  (Latency  -  2)  * 
(((Ln(Capacity+2))  -  2.20964)  *  -0.21 1 19)+  (Accuracy  -  75)  *  (((Ln(Capacity+2))  - 
2.20964)  *  0.012687) 
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Figure  A.2  examines  three  selected  simulation  runs  from  the  set  created  from  the 
simulation  experiments.  Three  outcomes  are  represented.  One  is  poor  (LR  ^9)  and  the  oth¬ 
ers  are  better  (LR  >  12).  The  poor  outcome  happens  to  be  one  where  the  capacity  was  se¬ 
verely  limited  (3).  The  better  outcomes  had  no  such  limitation.  We  infer  that  the  agent’s 
needs  for  messaging  exceed  a  hard  limit  of  3,  so  that  the  poor  case  was  a  result  of  poor  com¬ 
munication  capacity. 

Figure  A.2 

Simulation  Run  Results  That  Explain  the  Role  of  Capacity 
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Second  Set  of  Experiments:  Dynamic  Communication  Model 

Figure  A.3  is  a  plot  of  the  actual  data  versus  model-predicted  data.  Note  that  in  Figure  A.3 
and  similar  figures,  the  diagonal  lines  represent  best  fit  and  confidence  intervals. 

Figure  A.3 

Fit  Achieved  for  Modeling  Second  Set  of  Results 


LR  Predicted 
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The  equation  for  the  model  resulting  from  analysis  of  the  second  set  of  simulation 
runs,  as  well  an  assessment  of  the  relative  contribution  of  each  term  is  as  follows. 


LR  =  -72.62-0.4839*Capacity  -  2.0485*Latency-0.00667*(Capacity  -  56.3 1 58)*(Latency  - 

0.94737)  +  0.0369*Accuracy  -  0.004 12(Latency  -  0.94737)*(Accuracy-  75)  + 
0.000699*TargetRate  -  0.005 1 3* (Latency  -  0.94737)  T'larget  Rate  -  125)  + 
0.0672*(Sensing)-0.000842*(Capacity-  56.31 58)*(Sensing -  62.5)  -0.024  l*(Latency  - 
0. 947 37)  *  (Sensing  -  62.5)  +  0.000358*(Accuracy  -  75)*(Sensing  -  62.5)  + 
0.000104*(TargetRate  -  125)*(Sensing  -  62.5)  +  26.802*Ln(Capacity)  +  0.197*(Capacity  - 
56.3158)*(Ln(Capacity)  -  3.87424)  +  1.55(Latency  -  0. 94737) *(Ln (Capacity)  -  3.87424) 
-0.0189*(Accuracy-  75)*(Ln(Capacity)  -  3.87424)  +  0.0751*(Sensing  -  62.5) 
*(Ln(Capacity)  -  3.87424) 


Third  Set  of  Experiments:  Expanded  Three-Squad  Scenario 

Figure  A.4  is  a  plot  of  the  actual  versus  predicted  for  the  meta-model  synthesized  for 
experiment  3. 

Figure  A.4 

Fit  Achieved  for  Modeling  Third  Set  of  Results  (Two  Squads) 


LR=  -1.29651 1531  +  0.023468764  *  CapacitySqdl  +  0.027399798  *  CapacitySqd2  -  0.0002843 
*  (CapacitySqdl  -  43.3333)  *  (CapacitySqd2  -  43.3333)  -  0.990199934  *  Latency  + 
0.006961224  *  (CapacitySqdl  -  43.3333)  *  (Latency -2)  +  0.005752684  *  (CapacitySqd2 - 

43.3333)  *  (Latency -2)  +  0.042456186  *  Accuracy  -  0.00021383  *  (CapacitySqdl  - 

43.3333)  *  (Accuracy  -  75)  -  9.84688E-05  *  (CapacitySqd2  -  43.3333)  *  (Accuracy  -  75)  - 
0.003749573  *  (Latency  -  2)  *  (Accuracy  -  75)  +  0.002041734  *  Targeting  -  1.68459E-05  * 
(CapacitySqdl  -43.3333)  *  (Targeting  -  133.333)  -  8.33756E-06  *  (CapacitySqd2 - 

43.3333)  *  (Targeting  -  133.333)  -  0.002092867  *  (Latency -2)  *  (Targeting  -  133.333)  + 
1.12169E-05  *  (Accuracy-  75)  *  (Targeting-  133.333)  +  0.116404973  *  Sensing  + 
0.000394817  *  (CapacitySqdl  -  43.3333)  *  (Sensing  -  53.3333)  +  0.000491893  * 
(CapacitySqd2  -  43.3333)  *  (Sensing  -  53.3333)  -  0.019494371  *  (Latency  -  2)  *  (Sensing  - 

53.3333)  +  0.000738129  *  (Accuracy  -  75)  *  (Sensing  -  53.3333)  +  4.31927E-05  * 
(Targeting  -  133.333)  *  (Sensing  -  53.3333) 
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Fourth  Set  of  Experiments:  "Mosque-UAV"  Scenario 

The  Mosque  Direct-Fire  Only  Scenario:  Model  of  the  Loss  Exchange  Ratio 

Figure  A. 5 

Actual  Versus  Predicted  for  Model  of  Ln(LR)  for  Variant  1 


Ln(LR)  =  0.9327943  +  (RangerSensing  *  0.023402)  +  (UAVSensing  *  0.0022061)  + 

((RangerSensing-  30)  *  (UAVSensing  -  100)  *  0.0000042)  +  (Accuracy  *  0.0014909)  + 
((RangerSensing  -  30)  *  (Accuracy  -  75)  *  -0.000254)  +  ((UAVSensing  -  100)  * 
(Accuracy  -  75)  *  -0.000048)  +  (Latency  *  0.0186922)  +  ((RangerSensing  -  30)  * 
(Latency  -  2)  *  0.000565)  +  ((UAVSensing  -  100)  *  (Latency -2)  *  -0.000377)  + 
((Accuracy  -  75)  *  (Latency  -  2)  *  0.000305)  +  (ln(capacity)  *  -0.042042)  + 
((RangerSensing  -  30)  *  (ln(capacity)  -  3.5326)  *  0.003116)  +  ((UAVSensing  -  100)  * 
(In(capacity)  -  3.5326)  *  0.0001381)  +  ((Accuracy-  75)  *  (In(capacity)  -  3.5326)  * 
0.0005501)  +  ((Latency -2)  *  (In (capacity)  -  3.5326)  *  0.001937) 
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The  Mosque  Indirect-Fire  Scenario:  Model  of  the  Loss  Exchange  Ratio 

Figure  A. 6 

Actual  Versus  Predicted  for  Model  of  Ln(LR)  for  Enhanced  Blue 


Ln(LR)  Predicted 


Ln(LR)  -1.444054  +  (RangerSensing*0.005818)  +  (UAVSensing  *  0.0007559)  + 

((RangerSensing  -  30)  *  (UAVSensing  -  100)*4.5684e-9)  +  (Accuracy* 
0.0174631)+((RangerSensing -  30)  *  (Accuracy-  75)  *  -0.000112)  +  ((UAVSensing  - 
100)  *  (Accuracy  -  75)  *  -  0.000024)  +  (Latency  *  -  0.053379)  +  ((RangerSensing  -  30) 

*  (Latency- 2)  *-0.000031)  +  ((UAVSensing  -  100)  *  (Latency -  2)  * -0.000466)  + 
((Accuracy  -  75)  *  (Latency -2)  *-0.002877)  +  (Ln(Capacity)  *  0.1021319)  + 
((RangerSensing  -  30)  *  (Ln(Capacity)  -  3.5326)  *  0.00031 17)  +  ((UAVSensing  -  100) 

*  (Ln(Capacity)  -  3.5326)  *  0.0006046)  +  ((Accuracy  -  75)  *  (Ln(Capacity)  -  3.5326)  * 
0.0056856)  +  ((Latency  -  2)  *  (Ln(Capacity)  -  3.5326)  *  -0.01 1042) 

Note:  the  factor  “IFRange”  is  not  in  the  equation  because  it  proved  to  be  relatively  insignifi¬ 
cant. 


The  Mosque  Non-lndirect  Scenario  Fire  Model  of  the  Probability  of  Mission  Success 

Logit(Probability  of  successful  mission) = 

(-4.14229)  +  0.0035755  *  UAVSensing  +  0.0296077  *  Accuracy  +  ((UAVSensing  -  100)  * 
(Accuracy  -  75)  *  -0.000023)  +  -0.017885  *  Latency  +  ((UAVSensing  -  100)  *  (Latency  -  2) 
*-0.000807)  +  ((Accuracy  -  75)  *  (Latency  -  2)  * -0.004596)  +  0.1310321  *  (Ln (Capacity)) 
+  ((UAVSensing  -  100)  *  ((Ln(Capacity))  -  3.5326)  *  0.0014441)  +  ((Accuracy  -  75)  * 
((Ln(Capacity))  -  3.5326)  *  0.0096185)  +  ((Latency  -  2)  *  ((Ln(Capacity))  -  3.5326)  * 
0.0423696) 
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The  Mosque  Indirect  Scenario  Fire  Model  of  the  Probability  of  Mission  Success 

Logit(Probability  of  successful  mission)  = 

(-6.131673)  -  0.007759  *  RangerSensingJF  +  0.0008443  *  UAVSensingJF  + 
((RangerSensingJF  -  29.8857)  *  (UAVSensingJF  -  99.4286)  *  0.000078)  +  0.0458734  * 
AccuracyJF  +  ((RangerSensing_IF  -  29.8857)  *  (AccuracyJF  -  75.1429)  *  -0.000658)  + 
((UAVSensingJF  -  99.4286)  *  (Accuracy JF  -  75.1429)  *  -0.000135)  +  -0.26942 6  * 
LatencyJF  +  ((RangerSensingJF  -  29.8857)  *  (Latency JF  -  1.94286)  *  0.0005658)  + 
((UAVSensingJF  -  99.4286)  *  (LatencyJF  -  1.94286)  *  -0.001092)  +  ((Accuracy JF  - 
75.1429)  *  (LatencyJF  -  1.94286)  *  -0.007171)  +  0.7642031  *  LN (Capacity JF)  + 
((RangerSensingJF  -  29.8857)  *  (Ln(CapacityJF)  -  3.58755)  *  0.0014354)  + 
((UAVSensingJF  -  99.4286)  *  (Ln(CapacityJF)  -  3.58755)  *  0.0056196)  +  ((AccuracyJF 
-  75.1429)  *  (Ln(CapacityJF)  -  3.58755)  *  0.0200284)  +  ((LatencyJF  -  1.94286)  * 

(Ln (Capacity JF )  -  3.58755)  *  0.1741425) 


APPENDIX  B 


Past  Work  on  Communication  Models  Leveraged  in  This  Report 


In  the  body  of  the  report,  a  second-order  polynomial  was  used  to  represent  message  reliabil¬ 
ity  (or  message  completion  rate,  which  is  similar  to  packet  delivery  ratio).  The  use  of  a 
second-order  polynomial  was  based  on  results  from  earlier  work  by  Porche,  Jamison,  and 
Herbert  (2004).  A  brief  review  of  that  work  follows. 

A  number  of  simulation  experiments  were  performed  to  generally  assess  communica¬ 
tion  network  capabilities.  From  this  data,  a  model  was  fit.  Specifically,  a  factorial  design 
(Law  and  Kelton,  1991)  was  employed  for  rhe  factors  described  earlier  (transmission  fre¬ 
quency,  number  of  UAVs,  etc.)  and  shown  in  Figure  B.l. 

Figure  B.l 

Factors  and  Responses 


Factors 

Frequency 
_ Radio  Power 

Network  Density 
_ #  UAVs 

Radio  throughput  rate 
_ Data  rate 

_ LOS 

Distance  between  nodes 


Radio  Specifications 

A 


Network 

Simulator 

(Qualnet) 


Outputs 

Packet  Delivery  Ratio 
(percentage) 


Delay  (seconds) 


A  number  of  simulation  experiments  were  run.  Derails  are  shown  in  Table  B.l.  The 
table  shown  is  not  the  actual  experiment  designed  but  is  included  to  indicate  how  the  factors 
were  varied;  the  last  two  columns  represent  rhe  responses  sought,  which  are  PDR  (packet  de¬ 
livery  ratio  or  message  completion  rate)  and  end-to-end  delay.  The  simulations  involved  spe¬ 
cific  areas  of  interest. 
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Table  B.1 

Example  Design  of  Experiments 


# 

Frequency 

#  UAVs 

Density 

Radio 

Power 

Radio 

Throughput 

Data 

Rate 

PDR 

Delay 

1 

2  GHz 

0 

145 

2  W 

2Mbps 

16Kbps 

result 

result 

2 

1  GHz 

4 

72 

20  W 

2Mbps 

32Kbps 

result 

result 

3 

0.4  GHz 

8 

36 

1600  W 

2Mbps 

64Kbps 

result 

result 

4 

0.2  GHz 

0 

145 

2  W 

6  Mbps 

160Kbps 

result 

result 

5 

2.5  GHz 

4 

72 

20  W 

6  Mbps 

320Kbps 

result 

result 

6 

0.1  GHz 

8 

36 

1600  W 

6  Mbps 

533Kbps 

result 

result 

Specifically,  it  was  an  area  with  randomly  scattered  nodes  spread  across  a  25km  by 
25km  piece  of  terrain  in  the  Macedonia-Serbia  region.  Data  traffic  was  varied.  Network  per¬ 
formance  was  observed  as  function  of  frequency,  mobility,  UAV  usage,  etc.  The  terrain  is 
fairly  flat. 

All  of  the  experiments  were  20-minute  Qualnet  simulations.  These  experiments  fo¬ 
cused  on  data  rates.  A  predictive  model  of  the  packet  delivery  ratio  can  be  developed  from 
data  from  simulation  runs,  as  shown  in  Figure  B.2.  The  logit  function  is  convenient  because 
it  produces  a  value  between  0  and  1 .  It  is  defined  as  follows: 


l°git(/>)  =  log 


P 

wj 


Once  a  model  for  the  logit(PDR)  is  determined,  the  actual  PDR  (labeled  p )  may  be 
calculated  as  follows: 


exp(logit(^)) 
(exp(logit(^))  + 1 


Figure  B.2 

Generic  Form  of  Meta-Model 

Logit  (PDR)  =  p0  + 


P1(Fj-equency)+P2(UAVs)-l-. 


Other 
First-Order 
I  Terms 


Other 

P3(FrequencyxUAVs)+. . .  -  '  Second-Order 

Terms 


Other  Fligher  Order  Interactions 
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Figure  B.3 

Polynomial  Fit  Obtained  Based  on  Meta-Model  Data 


Area  2  (25  km  x  25  km)  Performance 
for  Nodes  lOKm  Apart 


— —  8  UAVs 
— —  0  UAVs 
a  8  UAVs,  freq=2.5GHz 
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